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following  topics:  spectral  analysis  via  combined  temporal  and  lag 
weighting;  programs  for  and  performance  of  multi-channel  linear  predictive 
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Spectral  Analysis 
Via  Quadratic  Frequency- 
Smoothing  of  Fourier- 
Transformed,  Overlapped, 
Weighted  Data  Segments 


A.  H.  Nuttall 

ABSTRACT 

A  generalized  framework  for  spectral  analysis  is  presented,  which 
includes  as  special  cases  the  Blackman-Tukey  technique  and  the 
weighted  overlapped  segment-averaging  FFT  technique.  The  general 
method  is  analyzed  in  terms  of  the  mean  and  variance  of  the  spectral 
estimate,  thereby  revealing  the  fundamental  dependence  of  its 
performance  on  the  temporal  weighting,  lag  weighting,  amount  of 
overlap,  number  of  pieces,  available  data  record  length,  and 
frequency  resolution.  To  enable  a  fair  tradeoff  study  and  comparison 
between  many  different  special  cases  of  the  technique,  it  is 
demanded  that  the  spectral  analysis  technique  achieve  a  specified 
frequency  resolution  with  the  given  data  record  length.  This 
necessitates  a  detailed  investigation  of  the  windowing  capabilities  of 
the  temporal  and  lag  weightings,  their  interaction,  and  the  definition 
of  an  overall  effective  weighting  and  window.  The  possibility  of  using 
lag-reshaping  to  achieve  desirable  effective  windows  is  considered 
and  found  to  be  reasonable  for  a  wide  variety  of  windows  with  good 
side  lobe  behavior  and  decay  rates. 

Results  for  the  variance  of  the  spectral  estimate  for  rectangular 
temporal  weighting  indicate  that  if  the  length  of  the  temporal 
weighting  is  selected  to  be  somewhat  larger  than  the  length  of  the  lag 
weighting,  the  variance  is  at  a  near  minimum.  Furthermore,  in  this 
situation,  the  possibly  deleterious  side  lobes  of  the  temporal 
weighting  can  be  compensated  by  proper  choice  of  lag  weighting, 
resulting  in  low  side  lobes  and  gopd  decay  of  the  overall  effective 
spectral  window.  For  Hanning  temporal  weighting,  the  lengths  of  the 
temporal  and  lag  weighting  should  be  approximately  equal  for 
minimum  variance  of  the  spectral  estimate. 
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Spectral  Analysis  via  Quadratic  Frequency-Smoothing 
of  Fourier-Transformed,  Overlapped,  Weighted  Data  Segments 

Introduction 

Spectral  analysis  techniques  have  received  a  great  deal  of  attention  in  the  past 
(references  1-12),  ranging  from  the  original  autocorrelation  approach  of  Blackman- 
Tukey  (reference  2)  to  the  more  recent  weighted,  overlapped,  segment-averaging 
FFT  approach  (references  7-12).  These  two  apparently  disparate  approaches  are 
shown  here  to  be  limiting  special  cases  of  a  generalized  framework  for  spectral 
analysis;  thus  consideration  of  this  general  technique  elucidates  the  fundamental 
behavior  and  performance  of  a  rather  wide  variety  of  spectral  approaches  and  their 
tradeoffs.  This  generalized  framework  has  already  been  presented  in  references  13- 
15,  where  a  brief  summary  of  some  of  the  main  features  has  been  mentioned. 
Additionally,  some  of  the  analytical  results  to  be  presented  here  were  alluded  to 
there;  however,  none  of  the  detailed  derivations  or  quantitative  results  in  this  report 
were  given  at  that  time. 

There  are  two  fundamental  parameters  that  critically  affect  the  performance  of 
any  spectral  estimation  technique.  They  are  the  available  record  length,  T,  of  the 
stationary  random  process  under  investigation,  and  the  effective  frequency 
resolution,  Be,  of  the  technique  under  consideration.  We  would  like  to  be  able  to 
attain  fine  resolution  (small  Be)  with  short  data  lengths  and  storage  (small  T); 
however,  stable  results  (small  fluctuations)  are  achievable  only  if  the  product  TBe  is 
much  larger  than  unity.  The  problems  we  address  are  how  to  make  optimum  use  of 
a  given  limited  amount  of  data  in  order  to  realize  a  specified  desired  resolution  wi  h 
maximum  stability,  and  to  determine  what  tradeoffs  are  available  regarding  win¬ 
dowing  and  weighting  at  different  stages  of  the  spectral  analysis  procedure.  It  is 
assumed  that  the  reader  is  familiar  with  the  tradeoffs  presented  in  reference  9  for 
the  weighted,  overlapped,  segment-averaging  FFT  procedure. 

The  generalized  framework  for  spectral  analysis  that  is  presented  here  is  capable 
of  a  wide  variety  of  forms  in  addition  to  the  Blackman-Tukey  and  FFT  approaches 
mentioned  above.  In  order  to  compare  these  various  forms  with  each  other  on  a 
reasonable  basis,  it  is  required  that  each  analysis  technique  realize  the  same  effective 
resolution  bandwidth,  Be,  and  that  they  all  utilize  the  same  data  record  length  T. 
Without  these  reasonable  constraints,  valid  conclusions  about  relative  per¬ 
formances  of  different  techniques  are  tentative  at  best.  This  insistence  upon  equal 
effective  frequency  resolution  necessitates  a  rather  detailed  investigation  of  the 
effects  of  the  weightings  and  windows  employed  in  the  generalized  framework  and 
their  allowed  durations.  The  desirability  of  an  overall  effective  window  for  spectral 
analysis  with  low  side  lobes  and  good  decay  is  achievable  only  through  careful 
choice  of  the  combined  weightings.  The  constraint  upon  the  effective  frequency 
resolution  naturally  also  shows  up  in  the  analysis  of  the  variance  of  the  spectral 
estimation  technique,  as  well  as  in  its  average  value,  leading  to  some  numerical 
analysis  complications;  nevertheless,  it  is  believed  to  be  the  proper  basis  of  com¬ 
parison  and  is  maintained  throughout. 
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The  two  major  statistical  parts  of  this  report  deal  with  the  mean  and  the  variance 
of  the  spectral  estimate.  The  .esult  for  the  average  value  leads  to  the  definition  of 
the  effective  window  of  the  generalized  spectral  analysis  technique,  in  terms  of  the 
temporal  and  lag  windows.  The  variance  result  incorporates,  additionally,  the 
amount  of  overlap,  the  number  of  data  pieces,  and  the  ambiguity  functions  of  the 
temporal  and  lag  windows;  the  complexity  of  the  latter  results  debilitates  easy  in¬ 
terpretation  and  it  has  been  found  necessary  to  resort  to  numerical  evaluation  of  the 
variance,  for  practical  cases  of  interest. 
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Ultimate  Stability  Attainable  From  a  Given  Record  Length 


Suppose  a  stationary  (complex)  data  record  x(t)  of  length  T  seconds  is  available, 
and  that  we  wish  to  estimate  its  power  density  spectrum*  G(f)  with  an  effective 
frequency  resolution  of  Be  Hz,  where  W0(f)  is  the  narrowband  window  through 
which  the  power  density  spectrum  is  to  be  observed.  These  two  frtquency-domain 
quantities  are  related  according  tot 


B 

e 


[Jdf  yf)]2 

Jdf  W*(f) 


(1) 


This  bandwidth  measure,  Be,  is  called  the  statistical  bandwidth  of  W0(f)  in  reference 
5,  page  265.  The  relation  of  effective  bandwidth  Be  to  half-power  bandwidth  Bh  is 
considered  in  appendix  A;  it  is  shown  that  for  good  windows,  the  ratio  of  the  two 
bandwidths  is  relatively  independent  of  the  exact  window  shape.  Thus  it  is  possible 
to  translate  results  to  other  bandwidth  measures  without  significantly  affecting  the 
essential  quantitative  aspects. 

If  we  take  the  original  data  record  and  pass  it  through  a  narrowband  linear 
(complex)  filter  with  power  transfer  function  equal  to  the  window,  |H(f)|2  =  W0(f), 
and  which  is  centered  at  a  frequency,  f0,  of  interest,  we  will  have  lost  no  relevant 
information  about  the  process  in  the  frequency  band  of  interest,  because  we  have 
filtered  out  information  of  no  use.  We  can  now  estimate  the  power  in  the 
narrowband  filter  output  process  and  use  it  as  a  measure  of  the  spectrum  of  the 
input  process  in  the  neighborhood  of  frequency  f0.  See  figure  1 . 


Let  z(t)  be  the  complex  output  process  from  the  narrowband  filter  when  excited 
by  the  available  T  seconds  of  data  x(t).  If  we  ignore  a  starting  transient  (i.e., 
assuming  T  »  1  /Be),  the  filter  output  power  estimate  in  the  band  of  width  Be  is 

P  -  j  f  dt  |z(t) |2  =  / dt  gCt)  |  z  (t) | 2  ,  (2) 


•For  brevity,  we  use  the  term  spectrum  rather  than  autospectrum  in  this  report, 
tlntegrals  without  limits  are  over  the  range  of  the  nonzero  integrands. 
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where  gate  function 


g(t) 


1/T  for  t  e  T 


0  otherwise 


(3) 


The  measure  of  stability  we  adopt  for  this  estimator,  and  for  the  others  to  follow, 
is  the  quality  ratio  defined  as  2 


0  -  Var(P)  _  P2  - 
Q  =  “ ? 


Av 


% 


T 


(4) 


where  Av(£)  and  Var(£)  denote  the  average  value  and  variance  of  P,  respectively, 
and  an  overbar  denotes  an  ensemble  average.  We  have  average  value 


Av(P).-  / dt  g(t)  |z(t)|2-  |z(t)|2  «  Rz(0)-  JdfG(f)  |H(f) 


9  G(fQ)  /df  |H(f)  i 2  ,  (5) 

assuming  that  filter-input  spectrum  G(0  does  not  vary  quickly  with  respect  to  Be,  in 
the  neighborhood  of  f0.  Rz(t)  is  the  correlation*  of  filter  output  process  z(t). 

Also,  we  have  mean  square  value 

3  //  dt  du  g(t)  g(u)  |  z (t)  | 2  |  z(u)  | 2  .  (6) 

Now  in  the  interval  T,  filter  output  z(t)  will  be  approximately  a  stationary  zero- 
mean,  complex,  analytic  Gaussian  process  for  small  Be;  filter  H(f)  has  filtered  out 
zero  and  all  negative  frequencies.  Then  fourth-order  moment 


z(t)z*(t)z(u)z*(u)  ?R2(0)  +  |R  ft  -  u)|2  . 

z  '  z  (7) 

There  follows  from  (4)  and  (6), 

Var(P)  *  ff dt  du  g(t)g(u)  |Rz(t  -  u)|2  =  f dt  *  (x)  |Rz(t)|2  , 

(8) 


where  gate-correlation t  of  function  g(t)  is 

♦  (t)  =  j dt  g(t)g(t  -  t) 

O  ^ 


(9) 


4 


•For  brevity,  we  use  the  term  correlation  instead  of  autocorrelation  in  this  report. 

tFor  stationary  processes,  we  let  R  denote  the  ensemble-average  correlation,  whereas  for  aperiodic 

nonrandom  functions,  we  let  4  denote  the  integral  correlation;  see  (5)  and  (9) 
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Since  the  gate-correlation  |g(r)  extends  over  ±T,  while  process  correlation  R2(t) 
extends  only  over  approximately  ±3/Be,  we  have,  via  Parseval’s  theorem  and  for 
TBe » 1 , 

Var(£)  *  *  (0)  Jdx|R2(T^2  -  I  /  df  G2(f) 


Y  J  df  | H (f )  |4  G2(f)  ?  y  g2(£0)  / df  !H(f) 


The  quality-ratio  measure  of  stability  is  then,  from  (4),  (5),  (10),  and  (1), 


I  /dfW!4  !  fd£  W2(f)  ^ 

"|df|H(f)|2J2  T[/dfWo(f)r  ^ 


This  is  the  limiting  (smallest)  value  of  Q  for  specified  frequency  resolution  Be  and 
available  record  length  T  when  TBe »  1.  No  other  spectral  procedures  can  improve 
on  it;  they  can  merely  approximate  it.  As  such,  (1 1)  is  the  benchmark  against  which 
other  procedures  can  be  compared,  under  the  condition  that  T  and  Be  are  equal  to 
those  values  for  the  various  procedures  under  consideration. 

The  normalized  quality  ratio  is  defined  as  Q  •  TBe.  Thus  the  normalized  quality 
ratio  can  never  be  smaller  than  unity,  which  value  can  only  be  approached  for  large 
TBe  through  proper  processing  techniques. 
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Description  of  Spectral  Analysis  Technique 

We  begin  by  defining  a  temporal  weighting  function  w,(t)  of  finite  duration  L,; 
that  is, 

w,(t)  0  only  for  |t|  <  1-/2 

1  7  11  1  (12) 

As  shown  in  figure  2,  temporal  weighting  w,(t)  is  real,  even,  and  peaked  at  the 
origin.  Although  this  presentation  is  couched  in  terms  of  continuous  functions,  we 
shall  show  shortly  that  it  includes  discrete  digital  processing  as  a  special  case. 


The  available  data  record  is  x(t)  for  0  <  t  <  T;  this  (complex)  random  process  is 
presumed  second-order  stationary  in  that  observation  interval.  We  shift  the  tem¬ 
poral  weighting  by  L,/2  +  pS  and  multiply  it  by  x(t)  to  generate  the  p-th  piece  of 
weighted  data: 

y  (t)  =  x (t)  w.  (t  -  — -  ps)  for  0  <  p  <  p  _  i  . 

P  1  \  1  /  ”  ”  (13) 

Here  p  is  an  integer;  if  shift  S  <  L,,  then  yp(t)  and  yp+  ,(t)  will  overlap  on  the  t-axis. 

The  first-stage  power  density  spectral  estimate  at  frequency  f  is  obtained  by 
averaging  the  magnitude-squared  value  of  the  Fourier  transform  of  data  piece  yp(t), 
over  a  total  of  P  pieces: 

a  i  P-1  I  e  I2 

Gi  (f)  *  d  £  ldt  exp(-i2rrft)  y  (t)  for  any  f  . 

1  P  pTo  |  3  P  I  (14) 

This  procedure  is  the  same  as  that  considered  in  reference  7  and  in  reference  9,  eqs. 
(2)  and  (3).  Since  x(t)  is  available  only  for  0  <  t  <  T,  we  prevent  the  weighting  in  (13) 
from  extending  beyond  that  interval;  mathematically  this  means  that  we  must  have 


Lj  ♦  (P  -  1)  S  <.  T 


(15) 
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An  alternative  interpretation  of  (14)  is  very  illuminating.  We  define  the  inverse 
Fourier  transform  of  (14)  as  the  first-stage  (autocorrelation  estimate;  there  follows 
immediately  at  delay  t, 

Rj(t)  -  Jdf  exp(i2irfx)  fi^f) 


-  f  ZQ  Jdt  ypM  -  T>  for  a11  T  •  (16) 

where  we  have  allowed  random  processes  x(t)  and  yp(t)  to  be  complex.  This  is 
recognized  as  the  average  of  the  sample  correlations  that  can  be  formed  at  delay  t, 
from  each  of  the  P  pieces  of  weighted  data  in  (13).  Since  temporal  weighting  w,(t)  is 
zero  for  jt|  >  L,/2  according  to  (12),  we  see  from  (13)  and  (16)  that 


Rj(t)  *  0  for  | t |  > 


(17) 


The  parameter,  t,  is  called  the  lag  domain  variable,  because  of  the  way  it  appears  as 
a  delayed  time  in  (16).  Equation  (16)  (and  those  to  follow)  is  true  for  all  t.  Both 
sides  of  (16)  are  zero  over  most  of  the  range  of  r;  nevertheless,  it  is  mathematically 
convenient  to  employ  the  equality  of  both  sides  of  (16)  for  all  t  in  various  trans¬ 
formations  below. 

The  second-stage  power  density  spectral  estimate  is  defined  as  a  frequency- 
smoothed  version  of  the  first -stage  result: 

G2(f)  -  Gjff)  e  W2(f)  «  / du  G1(u)W2(f  -  u)  , 

where  fe  denotes  convolution.  This  is  termed  quadratic  smoothing  since  it  is  done  in 
terms  of  power  quantities  rather  than  voltages.  Equation  (18)  is  the  desired  output 
from  the  generalized  spectral  analysis  technique  considered  here.  W2(f)  is  called  the 
lag  window,  for  reasons  to  be  given  below.  The  equivalent  statement  to  (18)  in  the 
lag  domain  is  obtained  by  Fourier  transforming  (18);  the  second-stage  correlation  is 

R2(t)  s  Jdf  exp(i27rfi)  G2(f)  =  Rj(x)  w2(t)  ,  ^ 

where  we  used  (18)  and  (16)  and  defined  the  Fourier  transform  pair 
w2(t)  =  J  df  exp(i2iifT)  W2(f)  , 

W2Cf)  s  J  di  exp(-i2frfT)  w2(t)  .  (20) 

w2(t)  and  W2(f)  are  both  real,  even,  and  peaked  at  their  origins.  Since  w2(t)  appears 
multiplicaiively  in  (19),  it  is  calltd  the  lag  weighting;  its  transform  W2(f)  is  the  lag 
window.  The  convention  adopted  throughout  this  report  is  that  multiplication  by  a 
function  in  the  t  or  t  domains  is  called  a  weighting;  the  counterpart  to  this  operation 
in  the  Fourier  transform  domain  (frequency  f  domain)  is  convolution  and  is  called 
windowing. 
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We  shall  let  lag  weighting  w2(t)  be  of  duration  2L2;  that  is, 


w2(t)  -  0  for  |t|  >  L2  .  (21) 

A  typical  plot  is  shown  in  figure  3;  the  reason  for  the  apparent  notational 
discrepancy  between  the  lengths  in  figures  2  and  3  will  become  clear  when  the  lag- 
domain  counterpart  of  temporal  weighting  w,(t)  is  encountered  later. 


We  have  already  observed  that  ft  ,(t)  is  zero  for  |t|>L,  in  (17).  Therefore,  it 
follows  from  (19)  and  (21)  that 

R2(t)  ■  0  for  | t |  >  min  (Lj,  L2)  .  ^ 2 ) 

However,  although  we  must  have  temporal  length  L,  <  T  (from  (15)  for  P  =  1),  there 
is  no  restriction  on  L2.  We  could  have  L2  larger  than  L,  and  T;  this  would  simply 
mean  that  we  would  be  lag-weighting  some  zero  estimates  of  ft,(T)  in  (19)  for  the 
larger  values  of  |t|.  Also  there  are  no  constraints  such  as  realizeability  on  the  lag 
weighting  or  window. 

For  example,  the  special  case  of  no  quadratic  frequency-smoothing  corresponds 
to 


W2(f)  »  6(f),  w2(t)  *  1,  L2  *  «  for  no  smoothing, 

(23) 

for  which  (18)  yields  62(f)  =  6,(0-  Thus  we  have  our  standard  first-stage  spectral 
estimate  (14)  as  a  limiting  case  of  the  generalized  spectral  analysis  technique.  On  the 
other  hand,  if  lag  window  W2(f)  were  broad  (small  L2),  there  would  be  a  significant 
amount  of  smoothing  taking  place  in  the  band  about  u  =  f  in  (18)  where  window  W2 
is  non  zero. 

There  is  no  inherent  limitation  on  the  relative  sizes  of  L,  and  L2  as  yet:  L2  can  be 
chosen  as  large  as  desired,  while  L,  is  subject  to  the  upper  bound  T.  However,  when 
we  specify  the  overall  effective  frequency  resolution  of  the  generalized  technique,  a 
relation  between  L,  and  L2  will  ensue. 
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Another  important  special  case  of  the  generalized  spectral  analysis  technique  is 
afforded  by  P  *  1,  w,(t)  ■  1  for  |t|  <  L,/2,  and  L,  =  T.  Then  (16)  and  (13)  indicate 
that  A  ,(t)  is  simply  the  sample  correlation  of  the  available  data  x(t)  of  length  T, 
while  A2(r)  in  (19)  is  a  weighted  version  of  A,(r)  for  |r|  <  L2.  But  this  is  precisely  the 
Blackman-Tukey  approach  described  in  reference  2;  the  choice  of  lag  weighting 
w2(t)  and  its  length  is  fully  discussed  there.  For  example,  if  w2(r)  =  (T-|t|)-'  for 
|t|  <  L2  <  T,  then  A2(t)  is  an  unbiased  estimator  for  |t|  <  L2;  see  reference  2,  page 
11. 

For  P>1  and  general  temporal  weighting  w,(t),  lag  weighting  w2(t),  and  overlap, 
a  wide  variety  of  processors  is  possible  via  the  generalized  framework  set  up  above. 
How  should  the  two  weightings  be  traded  off  against  each  other?  Can  the 
deleterious  effects  of  a  poor  or  preselected  temporal  weighting  be  undone  by  proper 
choice  of  lag  weighting?  Recall  that  none  of  these  techniques  can  hope  to  better  the 
quality-ratio  result  (11),  but  hopefully,  some  can  do  as  well,  with  less  computational 
effort  and  storage. 

A  related  procedure  to  the  one  presented  here  has  been  given  in  references  16  and 
17.  However,  neither  incorporate  overlapping,  and  the  fundamental  tradeoffs 
between  the  temporal  and  lag  weightings  were  not  studied.  Furthermore,  the  only 
frequency-smoothing  case  considered  was  a  rectangular  boxcar,  which ,  severely 
limits  the  potential  of  the  technique;  some  advantages  of  the  generalized  technique 
considered  here  will  become  apparent  at  a  later  stage.  For  the  time  being,  we  ob¬ 
serve  that  side  lobe  control  will  be  realized  by  a  mixture  of  temporal  weighting  and 
lag  weighting  (frequency  smoothing),  while  stability  will  be  achieved  by  a  com¬ 
bination  of  segment  averaging  and  frequency  smoothing  (lag  weighting). 

Discrete-Time  Processing 

All  the  functions  above  have  been  tacitly  assumed  no  worse  than  discontinuous; 
see  Figures  2  and  3  for  example.  However,  there  is  nothing  in  the  above  mathematics 
which  precludes  impulsive  behavior.  For  example,  suppose  the  temporal  weighting 
is  a  sum  of  N,  equispaced  impulses: 


wi(t)  *  \  Z  wim  6(t  -  °V 

a 


(24) 


where  {wlm}  is  a  finite  length,  real  sequence,  symmetric  about  m  =  0;  this  corres¬ 
ponds  to  discrete  sampling  of  waveform  x(t)  at  time  spacing  A,.  The  p  th  piece  of 
weighted  data  is,  from  (13), 

y  (t)  *  x(t)  At  £  wim  6(t  "  ~T  '  pS  "  mO 

E  V  4  -  t-  -  »s  -  m4t)  •  <25> 

where  weighted  sample 


pb 


+  mA 


(26) 


9 


The  first-stage  correlation  estimate,  (16),  becomes 


A 

R 


(t) 


r 


k 


Jlk  6(T  ‘  kV 


where  the  area  of  each  impulse  is  given  by 


A 

R 


lk 


(27) 


(28) 


and 


J(p) 

Rk 


L  y 


a 


P»m 


yp  ,m-k 


(29) 


The  last  quantity  is  the  sample  correlation  of  the  p-th  set  of  samples,  and  ft,*  is  their 
average  over  the  total  of  P  pieces. 

The  first-stage  spectral  estimate  is  the  Fourier  transform  of  (27)  as  usual: 


A  A 

Gj  (f)  =  At  2-1  Rlk  exp(-i2irfkat)  , 
k 


(30) 


which  is  finite  for  all  f  and  is  of  period  1/A,  in  f.  An  alternative  expression  is 
available  by  substitution  of  (25)  in  (14): 


P-1 


Vf)  ■  7  £  \  £  V  exp(-i2,foi  ) 

p«0  m  r 


(31) 


These  two  expressions  hold  for  arbitrary  f;  either  one  can  be  used  to  obtain  the 
first-stage  spectral  estimate.  If  we  restrict  our  calculations  of  interest  to  multiples  of 
some  frequency  increment  AF,  (31),  for  example,  specializes  to 


P-1 


vv  *  f  £n  \  l  v  «pc-i2»»qvF> 

p=0  m  r 


(32) 


where  q  is  an  integer.  At  this  point,  there  needn’t  be  any  relation  between  A,  and  AF; 
we  can  calculate  the  spectral  estimate  at  any  frequencies  we  please.  However,  a 
favorite  choice  for  ecu  mutational  purposes  is  to  choose  frequency  increment 


jJJ-  »  N  =  power  of  2 


to  get  the  special  digital  processing  result 


P-1 


A 

G, 


NA. 


1  ‘  4 


p=0 


At  £yDm  exP(-i2irracl/N) 
m  " 


(33) 


(34) 
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which  is  recognized  as  the  familiar  power  average  of  N-point  FFTs  of  weighted  data 
sequences.  All  the  impulsive  functions  in  (24)-(27)  have  dropped  out  of  first-stage 
spectral  estimates  (30)-(34). 

The  temporal  window  associated  with  impulsive  weighting  (24)  is  its  Fourier 
transform 


Wj(f)  -  At  £  wlm  exp(-i2irfmAt) 
in 


(35) 


Having  picked  an  impulsive  temporal  weighting  w,(t),  we  are  still  free  to  select  the 
lag  weighting  or  lag  window  as  we  please.  For  example,  for  any  lag  weighting 
function  w2(t),  (19)  and  (27)  yield  second-stage  correlation  estimate 


A 

R2(t) 


w2(kAt)  R1Jc  6(t  -  kAt)  =  At 


E 

k 


6(x  -  k&t) 


(36) 


The  corresponding  second-stage  spectral  estimate  is  the  Fourier  transform 

G2(f)  *  At  £  «2(kAt)  Rlk  exp(-i2irfkAt)  «  At  £  R2k  exp(-i2irfkAt)  , 

k  k 

(37) 

which  is  everywhere  finite  and  has  period  1/A,  in  f.  Evaluation  of  (37)  can  therefore 
be  confined  to  |f|  <  (2A,)-'. 

These  results  apply  for  general  lag  weighting.  A  specific  choice  is  the  lag  window 
with  N2  equispaced  nonzero  impulses: 

W2(f)  ■  L  W2n  5Cf  ‘  nV  • 

n  (38) 


Frequency  spacing  Af  need  not  be  related  to  time  spacing  A,  in  (24),  nor  to  frequency 
increment  AF  used  in  the  frequency  and  FFT  calculations  above  in  (32)-(34).  Also 
there  are  no  relations  between  the  real  symmetric  sets  of  numbers  {wlm}  in  (24)  and 
{W2n}  in  (38).  Substitution  of  (38)  in  (18)  yields  for  the  second-stage  spectral 
estimate 

Vf)*Af  E  W2nVf  -nV  * 

n  (39) 


which  is  a  local  average  (of  the  first-stage  estimates)  in  the  band  about  the 
frequency,  f,  of  interest.  Equation  (39)  is  a  discrete,  quadratic,  frequency¬ 
smoothing  operation.  In  fact,  (39)  holds  for  lag  window  (38)  and  any  temporal 
weighting  w,(t);  it  is  not  limited  to  the  discrete-time  form  (24). 

If  we  limit  our  calculations  of  £^(f)  to  multiples  of  frequency  increment  AF  as  in 
(32),  then  (39)  yields 

V’V  ■  4f  E  k2„  S(<>4f  -  "V  ; 
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we  can  use  (30)  for  ( %  on  the  right-hand  side.  Finally,  if  we  take  frequency  in¬ 
crement  (33)  and  frequency  spacing  Af  =  (MA,)*1,  where  integer  M  is  a  submultiple 
of  I^and  MA,  is  of  the  order  of  2L,,  the  FFT  results  of  (34)  can  be  employed  in  (40). 
More  will  be  said  later  on  the  choice  of  frequency  spacing  Af. 

The  variety  of  forms  available  at  different  stages  of  the  data  processing  illustrates 
a  great  deal  of  flexibility  in  exactly  how  the  available  data  x(t)  is  processed.  For 
example,  one  might  first  evaluate  <jj  via  FFT  procedure  (34).  Then,  since  (30)  can  be 
expressed  as 

■  ‘t  £  "lk  “PC-lMcq/N)  ,  (4)) 


it  follows  that  the  complete  nonzero  portion  of  correlation  sequence  {Alk}  is 
recoverable  from  the  set  of  numbers  )}£-■  if  N>2N,-1,  where  N,  is  the 

number  of  nonzero  weights  {wlk}  in  (24)  (see  reference  18).  On  the  other  hand,  for 
N<2N,-1,  the  inverse  FFT  of  {6,(-£j-)}£->  would  yield  A,k  only  for  |k|  <  N-N, 
(reference  18,  eq.  (15));  thus  the  central  values  of  A  lk  are  recoverable  from  6,.  Then 
second-stage  correlation  estimate 


w2(kAt) 


(42) 


follows  from  (36),  and  the  final  spectral  estimate  follows  from  (37).  The  lag 
weighting  samples  in  (42)  are  arbitrary;  thus  this  is  a  very  general  procedure  for 
obtaining  estimate  <J2(f)  at  any  f. 

The  relations  in  this  subsection  hold  for  arbitrary  values  of  t,  f,  and  q.  However, 
the  functions  of  t  are  impulsive,  and  are  zero  outside  limited  ranges,  while  the 
functions  of  f  and  q  are  periodic.  These  properties  should  be  utilized  in  any  com¬ 
puter  processing  technique  employing  these  forms.  Some  further  useful  properties 
and  interrelationships  of  the  sampled  lag  weightings  and  lag  windows  are  presented 
in  appendix  B. 
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Average  Value  of  Spectral  Estimate 

We  now  return  to  the  general  situation  for  both  the  temporal  and  the  lag 
weightings;  that  is,  we  do  not  presume  discrete  sampling  in  time  or  discrete 
smoothing  in  frequency.  From  (16)  and  (13),  the  mean  value  of  the  first-stage 
correlation  estimate  is 

Av{Rj (t)  }  *  J  /  dt  *(t)x*(t  -  t)  Wj  -  —  -  psj 
"  T  "  T"  “  pS^  *  x(t)x*(t  -  t)  ^(x)  *  R(t)  ♦j (t) 


•  w 


(43) 


where  R(t)  is  the  true  correlation  of  stationary  process  x(t),  and  where 

♦j(t)  •  J dt  Wj(t)  Wjft  -  t) 


(44) 


will  be  called  the  correlation  of  real  temporal  weighting  w,(t);  see  the  footnote  to 
(9).  We  have  not  presumed  process  x(t)  Gaussian;  relation  (43)  holds  for  any 
stationary  process  x(t). 

Since  the  first-stage  spectral  estimate  6,(0  is  a  linear  operation  (Fourier  trans¬ 
form)  of  ft  ,(t),  the  mean  value  of  6,(0  is  the  Fourier  transform  of  (43);  that  is, 

AvfGj (f) }  »Jdx  exp(-i2Trfx)  R(t)  ^(t) 


-  G(f)  9  wj(f)  *  f  du  Gj  (u)  wj(f  -  u)  ,  H5) 

where  G(0  is  the  true  spectrum  of  x(t),  i.e.,  Fourier  transform  of  R(x),  and  we  have 
Fourier  transformed  (44)  by  interchanging  integrals  and  using  temporal  window 


Wj(f) 


j  dt  exp(-i2nft)  w^t) 


(46) 


The  convolution  result  in  (45)  is  a  familiar  one  for  the  standard  FFT  processing  of 
weighted,  overlapped  data  segments;  see  reference  9,  eq.  (5),  for  example.  Window 
W,(0  is  real  and  even  about  f  =  0,  since  weighting  w,(t)  is  real  and  even  about  t  =  0. 

The  mean  value  of  the  second-stage  correlation  estimate  follows  immediately 
from  (19)  and  (43): 

Av{R2(x)}  =  R(x)  $ x ( t)  w2(x)  =  R(x)  we(x)  , 


where 
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«eCO  H  ♦i(t)  w2(t)  (48) 

is  called  the  effective  (overall)  weighting  of  the  generalized  spectral  analysis 
technique.  It  incorporates  the  temporal  weighting  w,(t)  through  its  correlation  4,(r) 
defined  in  (44),  and  it  involves  lag  weighting  w2(t)  directly.  Reference  to  (44)  and  to 
figure  2,  for  a  typical  temporal  weighting,  shows  that  +,(t)  is  as  depicted  in  figure  4; 
|,(t)  extends  over  (-L,,  L,)  and  is  zero  for  |t|  >  L,.  Since  the  effective  weighting 
we(r)  in  (48)  involves  +,(t)  and  w2(r),  we  now  see  the  reason  for  the  particular 
choices  of  L,  and  L2  in  figures  2-4.  Specifically,  ±L,  and  ±L2  measure  the  non¬ 
zero  extent,  in  the  r-domain,  of  the  functions  that  are  relevant  to  the  effective 
weighting.  Although  L,  measures  the  nonzero  extent  of  temporal  weighting  w,(t)  in 
the  time  domain  in  figure  2,  and  the  nonzero  extents  of  |,(r)  and  w2(t)  are  2L,  and 
2L2  in  figures  4  and  3,  respectively,  we  will  nevertheless  refer  to  L,  and  L2  as  the 
“lengths”  of  4,(r)  and  w2(r),  respectively,  in  the  r-domain,  for  convenience. 


In  appendix  C,  4,(r)  is  evaluated  for  the  class  of  temporal  weightings* 

w,(t)  =  £  a.  exp(i2Trkt/L1 )  for  |t|  <  L./2  , 

1  ]<  *  1  1  (49) 

which  includes  a  wide  variety  of  weightings  such  as  rectangular,  Hamming, 
Hanning,  Blackman,  Harris,  and  the  recent  optimal  weightings  of  Nuttall,  reference 
19.  Specializations  to  real  symmetric  {ak}  and  to  a  limited  number  of  nonzero 
coefficients  are  also  made  in  appendix  C. 

Finally,  since  second-stage  spectral  estimate  62(f)  is  a  Fourier  transform  of  A2(t), 
its  mean  value  follows  from  (47)  and  (48)  as 


Av{G2(f)}  =  G(f)  ©  Wg(f) 


(50) 


where 


W 

e 


(f) 


J dx  exp(-i2irfr)  wg(T)  =  W^(f)  Q  W2(f) 


(51) 
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*For  brevity,  here  and  later,  we  omit  the  "0  otherwise”  statement  that  applies  for  |t|  >  L , /2.  as  was 
done  in  (3). 


is  the  effective  (overall)  window  of  the  generalized  spectral  analysis  technique  of 
interest  here.  The  result  in  (51)  follows  by  Fourier  transformation  of  the  product  in 
(48)  and  use  of  (44)  (just  as  done  in  (45)).  Relation  (50)  is  a  simple  and  informative 
one  for  the  average  spectral  estimate;  it  enables  ready  determination  of  the  amount 
of  spreading  caused  by  particular  choices  of  temporal  and  lag  windows.  It  holds  for 
any  stationary  process  x(t)  with  spectrum  G(f);  thus  x(t)  needn’t  be  a  Gaussian 
process  for  (50)  to  hold  true. 

As  a  special  case  of  (50),  consider  lag  weighting  w2(t)  to  be  1  for  all  r.  Then 
W2(f)  *  <5(0  and  (50)  reduces  to  the  result  in  (45)  as  expected,  since  we  are  em¬ 
ploying  no  lag  weighting  at  all  in  this  case. 

As  another  special  case,  let  temporal  weighting  w,(t)  be  1  for  all  |t|  <  L,/2  and  let 
L,  *  T,  L2 «  T.  This  corresponds  to  Blackman-Tukey  processing.  Then  Wf(f)  is 
proportional  to  sinc^CTf).  which  is  much  narrower  in  f  than  W2(f),  meaning  that 
We(f)  ~  W2(f),  the  lag  window  alone. 

Interpretation  of  the  response  of  the  effective  window,  We(f),  via  convolution 
(51)  can  sometimes  be  deceiving,  and  it  may  be  helpful  and  necessary  to  resort  to 
(48).  For  example,  suppose  w2(t)  is  1  for  |t|  <  L2  and  0  otherwise,  where  L2  >  L,. 
Then  (51)  says  that  we  have  to  convolve  sinc(L2f),  which  has  -6.63  dB  side  lobes, 
with  Wf(f).  Our  first  impression  would  be  that  We(f)  is  bound  to  have  bad  side  lobes 
regardless  of  the  temporal  window.  But  recourse  to  (48)  and  figure  4  immediately 
reveals  that  we(r)  *  f  ,(t)  for  all  r,  and  that  W2(f)  is  totally  irrelevant,  provided  that 
L2  >  L,.  The  scaling  of  4,(t)  by  a  constant  in  (48),  over  the  range  of  nonzero  $,(t), 
obviously  has  no  effect  on  the  relative  side  lobes  of  We(f).  Furthermore,  the  actual 
calculation  of  the  effective  window  via  (51)  is  often  tedious,  whereas  a  Fourier 
transformation  of  the  product  in  (48)  is  a  reasonable  approach,  even  if  only  by  an 
FFT. 


Constraint  on  Temporal  and  Lag  Weighting  Lengths 

The  effective  window  We(f)  was  presented  in  (51).  Its  “width”  is  given  ap¬ 
proximately  bv  the  sum  of  the  widths  of  the  temporal  and  lag  windows.  As 
discussed  earlier,  we  wish  to  constrain  the  effective  bandwidth  Be  of  We(f),  so  as  to 
be  able  to  fairly  compare  the  performance  of  different  spectral  analysis  techniques. 
The  effective  bandwidth  is  given  by  (1)  and  can  be  developed  as 


where  we  have  used  Parseval’s  theorem,  the  Fourier  transform  relationship  in  (51), 
and  (48).  Since  Be  is  to  be  considered  fixed,  (52)  forces  a  relationship  between 
lengths  L,  and  L2  of  +j(t)  and  w2(t). 

For  example,  consider  rectangular  temporal  and  lag  weightings  (this  is  not  a 
practical  case  and  is  presented  only  for  illustration  purposes): 

wx(t)  -  1  for  | t |  <  Lj/2  , 

♦l(T)  -  4  -  M  for  | t |  <  Lj  , 


w2(t)  *  1  for  | t |  <  L2 

Then  (52)  yields 


where 

Lffl  =  minfLj,  L2) 


(53) 


(54) 


(55) 


Given  a  value  of  Be,  (54)  can  be  considered  as  an  equation  for  L2  in  terms  of  L,,  or 
vice  versa.  Here  we  have  fixed  the  shapes  of  the  weightings  and  are  varying  the 
lengths  so  as  to  realize  the  specified  frequency  resolution  Be. 
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Generally,  the  qualitative  character  of  the  interrelationship  between  lengths  is  as 
depicted  in  figure  5,  for  fixed  weighting  shapes.  The  larger  one  of  the  lengths  is 
made,  the  smaller  the  other  length  can  get  and  still  satisfy  the  specified  bandwidth 
of  the  effective  window.  For  a  specified  pair  of  shapes  for  w,(t)  and  w2(t),  a  plot 
like  figure  5  can  be  used  in  two  different  ways.  If  we  pick  a  value  for  B^L,,  this 
determines  B^  and  hence  L2/L,.  On  the  other  hand,  choice  of  a  value  for  the  ratio 
of  lengths,  L2/L,,puts  a  line  through  the  origin  of  slope  L2/L,,  and  thereby 
determines  BeL,  and  where  the  line  intersects  the  curve.  We  note  therefore  that 
knowledge  of  one  of  the  following  three  quantities  determines  the  other  two:  L2/L,, 
BgL,,  B{L2. 

The  limiting  parameter  values  on  figure  5  are  determined  as  follows:  as  L,  -*  », 
then  L2  -*  L2(min),  where  now  (from  (52)) 


Here,  c{  }  is  a  "shape  factor”  defined  for  any  limited-duration  function  g  as 


where  it  is  assumed  that 

g(-r)  =  0  for  | t [  >  Lg  »  (58) 


Figure  5.  Interrelationship  of  Lengths  L,  and  L2  for  Fixed  Shapes 
of  the  Temporal  and  Lag  Weightings 
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that  is,  2L,  is  the  nonzero  extent  of  g(r).  Shape  factor  c{g}  is  independent  of  the 
magnitude  of  g(r)  and  of  its  length  on  the  r-scale.  Thus  (56)  yields 
BjL^min)  «  (2  cjwj})*1,  which  is  entered  on  the  ordinate  in  figure  5.  However, 
since  Lj  is  limited  by  T,  the  dotted  portion  of  the  curve  on  the  far  right  is  not  at¬ 
tainable. 


Conversely,  if  instead,  L2  »,  then  L,  -*  L,(min),  where  now  (from  (52)) 


51  ■  U 


♦  jCt) 

/dr 

JJoT 

12 


2L1(min)  cf^} 


(59) 


that  is,  BgL^min)  =  (2  c{4,})-'.  This  value  is  attainable;  it  corresponds  to  no  lag 
weighting.  The  ratio,  L2/L,,  of  weighting  lengths  can  take  values  in  the  range 
L2(T)/T  to  »;  for  BeT  »  1,  this  constitutes  the  range  from  almost  zero  to  infinity. 

Since  the  shape  factors  c{w2}  and  c{+,}  are  important  limits  on  the  weighting 
lengths,  tables  of  their  numerical  values  for  a  number  of  useful  weight  functions  are 
given  below.  The  weightings  listed  under  C5,  C3,  Cl  are  those  given  in  reference  19, 
figures  10, 1 1, 12,  respectively;  the  notation  means 

C5:  continuous  fifth  derivative  of  weighting 
C3:  continuous  third  derivative  of  weighting 
Cl:  continuous  first  derivative  of  weighting 


For  the  class  of  lag  weightings  given  by 

w  (t)  »  £  a,  cos(Trkx/L_)  for  |t|  <  L 
k>.0  K  i 

the  shape  factor  is 


This  is  evaluated  for  several  weightings  in  table  1. 


Table  1.  Shape  Factor  for  Lag  Weighting  w2(r) 


W2(t) 

c{w2(t)} 

Rectangular 

1.000 

Hanning 

.3750 

Hamming 

.3964 

Blackman 

.3046 

C5 

.2256 

C3 

.2442 

Cl 

.2558 
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For  the  class  of  temporal  weightings  given  by 

w-  (t)  ■  £  av  cos(2»kt/Lt)  for  |t|  <  L./2  , 

k^JD  1  1 


(63) 


the  correlation  |,(t)  is  evaluated  in  general  in  appendix  C.  The  shape  factor  of  $,(r), 


can  then  be  evaluated  numerically  and  is  given  in  table  2. 

Table  2.  Shape  Factor  for  Correlation 
f  ,(t)  of  Temporal  Weighting  w,(t) 


W,(t) 

c{*,} 

Rectangular 

.3333  =  1/3 

Hanning 

.2405  *  (8n2  +  35)/(48n2) 

Hamming 

.2628 

Blackman 

.2073 

C5 

.1545 

C3 

.1678 

Cl 

.1763 

Plots  of  the  relationship  between  L,  and  L2  dictated  by  (52)  are  given  in  figure  6 
for  various  combinations  of  temporal  and  lag  weightings.  For  a  rectangular  lag 
weighting,  the  curve  will  actually  reach  BeL,  =  (2c{^1})*‘  when  L2  =  L,;  then  the 
curve  goes  vertically  up  from  this  point  for  L2  >  L,  (see  figure  6A).  The  procedure 
for  the  evaluation  of  figure  6  is  as  described  under  figure  5;  namely,  pick  a  value  for 
L2/L,,  compute  BeL,  via  (52),  and  then  compute  BeL2  =  BeL,*(L2/L,). 

If  the  maximum  segment  length,  L,,  is  specified  (as  for  example,  when  the 
maximum  FFT  size  and  the  time-sampling  spacing  A,  are  fixed),  under  what  con¬ 
dition  can  a  desired  effective  frequency  resolution,  Bd,  be  met?  The  answer  to  this 
question  is  available  from  figure  5;  namely,  we  see  that 


-1 


BgLj  >  (2c{^»  ,  or 


Be  “  2c )LX 


(65) 


I  ius  if  desired  resolution  Bd  is  greater  than  or  equal  to  the  right-most  term  of  (65), 
there  exists  a  choice  for  lag  length  L2  that  will  yield  the  desired  frequency  resolution. 
The  shape  factor  in  (65)  depends  only  on  the  temporal  weighting  w,(t). 
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0  1  2  3  4  5  6 


6A.  Rectangular  Temporal  Weighting,  Rectangular  Lag  Weighting 


0  1  2  3  4  5  6 


6B.  Rectangular  Temporal  Weighting,  Hanning  Lag  Weighting 
Figure  6.  Allowed  Lengths  of  Various  Temporal  and  Lag  Weighting  Pairs 
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Figure  6.  Allowed  Lengths  of  Various  Temporal 
and  Lag  Weighting  Pairs  (Cont’d) 
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6E.  Rectangular  Temporal  Weighting,  Cl  Lag  Weighting 


6F.  Hanning  Temporal  Weighting,  Hanning  Lag  Weighting 


Figure  6.  Allowed  Lengths  of  Various  Temporal 
and  Lag  Weighting  Pairs  (Cont’d) 
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Normalization  of  Weightings 

The  average  of  the  first-stage  correlation  estimate  was  given  in  (43).  For  t  =  0,  it 
yields 


AvtRjfO) }  •  R(0)  ♦jCO) 


(66) 


Since  R(0)  is  the  true  power  in  the  process  x(t)  under  investigation,  it  is  convenient 
to  normalize  according  to 


1  ■  $j(0)  •  j  dt  Wj(t) 


(67) 


Then  A,(0)  is  an  unbiased  estimator  of  R(0). 

Additionally,  from  (47)  and  (48),  we  have,  for  the  second-stage  correlation 
estimate. 


Av{R2(0)}  -  R(0)  ♦jCO)  w2(0)  . 


(68) 


Therefore,  in  addition  to  (67),  we  also  set  lag  weighting  value 


w2(0)  -  1 


(69) 


making  A2(0)  an  unbiased  estimator  of  R(0).  There  follows,  for  the  effective 
weighting, 


w  (0)  *  1 
e 


(70) 


Since  there  is  no  significant  loss  of  generality,  the  normalizations  in  (67),  (69),  and 
(70)  will  be  used  in  the  rest  of  this  report. 


Discrete-Time  Processing 

For  the  impulsive  temporal  weighting  introduced  in  (24)-(29),  the  normalization 
(67)  must  be  modified  somewhat,  since  the  integral  of  wf(t)  in  (24)  would  be  infinite. 
We  resort  to  (28)  and  require  that  the  origin  value  of  the  sample  correlation  satisfy 
the  unbiased  requirement  that 

Av{R1q}  =  R(0) 

Reference  to  (28),  (29),  (26),  and  (43)  yields 

Av(R10}  =  AvfR^}  =  At  £  |y  I2  =  A 

m  r 


(71) 


L  S, R(0)  • 


(72) 


m 
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therefore  the  normalization  is 


A  5^  w?  «  1 

1  m  lm  (73) 

This  is  the  discrete  analog  to  the  integral  constraint  in  (67). 

The  correlation  +,(t)  of  temporal  weighting  w,(t)  in  (24)  is  given  by  (44)  as  usual 
and  is  expressible  as 


where 


♦j(t)  ■  At  Z)  $lk  “  kAt) 

♦lk  E  At  £  wl,m  wl,m-k  ‘ 
in 


(74) 


(75) 


Thus  we  see  that  (73)  is  tantamount  to  +10  =  1 ,  which  replaces  the  constraint  <f,(0)  =  1 
in  (67)  for  the  continuous  temporal  weighting  case. 


If  we  also  require  that  second-stage  correlation  estimate  (42)  satisfy  the  unbiased 
requirement 


Av{R20)  -  R(0) 


(76) 


then,  as  before,  we  require 

w2(0)  -  1 


(77) 


Finally,  the  effective  weighting  becomes,  upon  use  of  (74), 

w  (t)  *  $,(1)  w  (t)  =  A  Y,  tiv  w?(kO  6(t  -  kAJ  • 
e  11  tk  ^t  1  (78) 

The  normalizations  adopted  above  make  the  area  of  the  impulse  at  t  =  0  equal  to 


\  *10  “2(0) 


(79) 
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Example  *  of  Effective  Windows 

We  consider  first  a  rectangular  temporal  weighting  w,(t),  for  which  the 
correlation  is  triangular. 


♦jCt)  -  1  -  ItJ/Lj  for  | t |  <  Lj  , 


(80) 


and  the  class  of  lag  weightings  as  given  earlier  by  (61): 

w.(t)  ■  53  av  c°s(irkT/L.)  for  |t|  <  L 
1  kiO  K  1 

Then  constraint  (52)  yields 


B 

e 


cos(vkx)  1 


■]•  [■ 


(81) 


l2  <  4 


(82) 


The  effective  weighting,  we(r),  is  given  by  the  product  of  (80)  and  (81);  its  Fourier 
transform  is  the  effective  window 


i 


where 


v  =  B  L,  u  ,  u  =  f/B 
e  2  e 


(84) 


Although  (82)  and  (83)  could  be  extended  to  the  case  where  L2  >  L,,  that  range  is  not 
of  practical  interest,  as  will  become  apparent  later. 

The  numerical  procedure  for  evaluation  of  the  effective  window  is  to  first  select 
the  shape  of  the  lag  window  by  specifying  coefficients  {ak}.  Then  we  choose  a  value 
for  L2/L,  and  compute  BeL2  from  (82).  We  can  then  employ  (83)  and  (84)  to 
determine  We(f).  Four  examples  are  given  in  figure  7,  where  we  have  plotted 


W  (f) 

dB  0  10  log  wToT  vs 


(85) 
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and  Hanning  Lag  Weighting 


7B.  Effective  Window  for  Rectangular  Temporal  Weighting 
and  C5  Lag  Weighting 


Figure  7.  Examples  of  Effective  Windows  for  Rectangular  Temporal  Weighting 
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The  curve  corresponding  to  L2/L,  ■  0  (i.e.,  L,  =  «)  is  that  for  the  lag  window 
alone; 

The  overriding  impression  of  the  plots  in  figure  7  is  that  the  effective  window  has 
poor  side  lobe  behavior  and  decay  unless  L2/L,  is  chosen  very  small.  That  is,  the 
poor  side  lobe  behavior  of  temporal  window  Wf(f)  enters  the  convolution  (51)  for 
We(f),  and  is  difficult  to  suppress,  even  by  choice  of  good  lag  windows.  It  would  be 
desirable  to  realize  the  bottom-most  figures  in  each  of  these  plots,  since  these  latter 
curves  have  good  side  lobes  and  decay;  a  procedure  for  accomplishing  this  goal  is 
presented  in  the  next  section. 

The  situation  is  significantly  improved  when  the  temporal  weighting  is  tapered. 
An  example  for  Hanning  temporal  and  lag  weightings  is  given  in  figure  8.  The 
bottom-most  curve  has  an  eventual  18  dB/octave  decay  because  +,(t)  has  a 
discontinuous  fifth  derivative  at  r  *  0,  which  is  not  compensated  by  w2(t).  ($,(t) 
also  has  a  discontinuous  fifth  derivative  at  r«  ±L,,  but  this  is  converted  to  a 
discontinuous  seventh  derivative  for  we(r)  by  means  of  w2(t)  when  L2  =  L,.) 


Figure  8.  Effective  Window  for  Hanning  Temporal  Weighting  and 
Hanning  Lag  Weighting 
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Discrete- Time  Processing 

The  temporal  weighting  w,(t)  is  given  by  (24),  and  its  correlation  ^,(r)  by  (74), 
where  coefficients  {+lk}  are  given  by  (75).  The  Fourier  transform  of  (24)  leads  to 
temporal  window 


w.(f)  «  6  £  W  exp  (-i2irfmAt) 

■  (86) 

which  has  period  1/A,  in  f,  and  is  real  and  even,  since  weight  sequence  {wlm}  is  real 
and  even. 

For  a  general  lag  weighting  w2(r),  the  effective  weighting  is  given  by  (78).  The 
effective  window  We(f)  is  given  by  (5 1)  as  the  Fourier  transformation  of  (78): 

We(f)  -  At  £  *lk  w2(kAt)  exp(-i2rrfkAt)  ,  (g?) 

which  also  has  period  1/A,  in  f,  and  is  real  and  even,  since  lag  weighting  w2(t)  is  real 
and  even.  This  result  holds  for  any  lag  weighting  w2(r)  and  is  a  very  useful  form  for 
computing  We(f)  for  any  value  of  frequency  f.  The  convolutional  form  of  (51)  is  not 
very  useful  for  computing  We(f)  for  general  W2(f). 

As  a  special  case,  we  can  evaluate  (87)  at  particular  frequencies 
fn  *  nAp  *  n/(NA,)-',  as  in  (33)  and  (34): 

",  (>£7)  *  4t  £  *lk  "2(kV  •  (88) 

which  can  be  accomplished  as  an  N-point  FFT.  We  should  choose  N  large  in  order 
that  (88)  be  capable  of  tracing  the  fine  detail  of  We(f).  This  is  an  attractive  and 
efficient  way  to  evaluate  the  effective  window. 

A  Special  Lag  Window  for  Discrete-Time  Processing 

The  result  in  (87)  applies  for  discrete  time  sampling  and  arbitrary  lag  weighting. 
We  now  specialize  to  the  lag  window  given  in  (38): 


v£>  ■  4f  1:  h2„  s(f  -  "V  -  m 

where  sequence  { W2n}  is  real  and  even,  and  frequency  spacing  Af  need  not  be  related 
to  time  spacing  A,  in  (24).  Then,  via  inverse  Fourier  transformation  (20),  the 
corresponding  lag  weighting  is 


w2(t)  *  A^  £  exP(^7rnA^T) 

n 


and,  in  particular,  sample  values 

w2(kAt)  =  A^  ^  W2n  exp(i27ink  A^  At) 
n 


(90) 


(91) 
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which  can  be  used  in  (87)  to  evaluate  We(f)  for  any  frequency  f  whatsoever. 

An  alternative  form  to  (87)  for  calculation  of  We(f)  for  this  special  lag  window 
(89)  is  afforded  by  substitution  of  (89)  in  the  convolutional  form  of  (51): 


V«  ■  Af  £  H2n  "fc  -  "V  •  (W) 

The  temporal  window  Wj(f)  is  given  here  by  (86).  Equation  (92)  is  an  attractive 
form  when  the  number,  N2,  of  nonzero  coefficients  { W2n}  is  small  and  W  ,(f)  can  be 
evaluated  in  closed  form.  In  fact,  (92)  actually  holds  for  any  temporal  weighting 
w,(t);  it  is  not  limited  to  the  discrete-time  forms  (24)  and  (86).  Equations  (87)  and 
(92)  are  duals  in  the  sense  that  (87)  applies  to  any  w2(t)  and  an  impulsive  w,(t), 
whereas  (92)  applies  to  any  W,(f)  and  an  impulsive  W2(f).  Either  equation  can  be 
evaluated  at  any  f  of  interest. 

Our  first  example  is  rectangular  temporal  weighting;  from  figure  2,  (24),  and 
(73),  wlm  *  Lj1/2,  where  L,  »  N,A,.  Then,  from  (86), 


Y«  ■  ij 


sinfwLjf) 


Nj  sinOI^f/Nj) 


1  Li  yhf) 


(93) 


For  the  lag  window,  we  take  impulsive  form  (89);  then  (92)  and  (93)  give 


V«  ■li‘fS  w2„  v  <Lif  -  nLiaf>  ■ 


Two  important  choices  yet  to  be  made  are  L,Af,  the  relative  frequency  spacing  used 
in  frequency  smoothing,  and  the  set  of  coefficients  {W2n}.  For  Hanning  frequency 
smoothing,  the  latter  is 

11/2  for  n  -  0 

1/4  for  n  =  +1 
— 

0  otherwise 

The  effective  windows  for  L,Af  =  1/2  and  1  are  given  in  figure  9  for  N,  =  32. 
Window  (94)  is  even  about  f  =  0  and  has  period  1/A,  in  f;  hence  only  the  region 
0,  (2A,)'1  is  plotted  in  f.  The  window  in  figure  9A  has  no  deep  notches  since  the 
frequency  displacement  (spacing)  Af  =  (2L,)-'  causes  the  notches  to  be  filled  in;  the 
window  for  Af  =  Lj‘  in  figure  9B  reinforces  the  notches  and  has  a  significant 
shoulder  near  f  =  1/L,.  Both  windows  have  slow  decay  with  frequency  and  do  not 
have  significant  rejection,  even  near  Nyquist  frequency.  Closer  spacings  than 
(2L,)-‘  do  not  improve  the  decay  or  rejection  capabilities;  wider  spacings  than  Lj1 
generate  humps  in  the  effective  window.  The  bad  features  of  rectangular  temporal 
weighting  are  not  undone  by  Hanning  frequency  smoothing;  see  also  figure  7A. 
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Figure  9.  Effective  Window  for  Discrete-Time  Rectangular  Temporal 
Weighting  and  Hanning  Frequency-Smoothing 
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The  second  example  is  rectangular  temporal  weighting  with  N,  =  32  and  rec¬ 
tangular  frequency  smoothing  over  five  frequency  samples;  i.e., 


1/5  for  |n |  <.  2 
0  otherwise 


(96) 


The  effective  windows  for  L,Af  *  1/2  and  1  are  given  in  figure  10.  The  main  lobe 
humps  in  figure  10B  are  caused  by  the  displacements  of  Wf(f)  according  to  (92). 
Both  windows  again  have  poor  decay  tnd  poor  rejection;  however,  the  main  lobe  is 
more  box-like  in  shape  than  previously. 


The  third  example  is  identical  to  the  previous  one  except  that  N2  =  11.  The  plots 
in  figure  1 1  reveal  that  the  main  lobe  is  quite  box-like,  but  the  decay  and  rejection 
are  no  better  than  previous  cases.  According  to  (92),  we  are  merely  taking  the  poor 
side  lobes  and  decay  of  WJ(f)  and  moving  them  about,  but  not  improving  them  in 
any  way. 

The  last  example  in  this  subsection  is  Hanmng  temporal  weighting  with  no 
frequency  smoothing  at  all.  The  effective  window  for  N,  *  32  is  simply  Wf(f)  and  is 
plotted  in  figure  12.  It  has  the  familiar  -31.5  dB  peak  side  lobe,  a  rapid  decay,  and 
significant  rejection  capability. 
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Figure  11.  Effective  Window  for  Discrete-Time  Rectangular  Temporal  Weighting 
and  Rectangular  Frequency-Smoothing  with  N2  =  11 
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Lag  Reshaping  for  Desired  Effective  Windows 

From  (48),  the  effective  weighting  of  the  generalized  spectral  analysis  technique  is 
given  by 


we(x)  «  tjCt)  w2(t) 


(97) 


where  +,(t)  is  the  correlation  of  temporal  weighting  w,(t);  see  (44).  Now  suppose 
that  for  a  given  temporal  weighting  w,(t),  with  associated  correlation  $,(t),  we 
choose  lag  weighting 


wd(x) 

VT)  -  for 


M 


“  L2  -  L1 


(98) 


where  wd(r)  is  a  desirable  weighting  with  wd(0)  =  1  (in  keeping  with  (67)  and  (70)) 
and 


wd(t)  ■  0  for  | t |  >  L2  . 

(99) 

Notice  that  L2  >  L,  is  disallowed  in  (98)  since  |,(t)  =  0  for  |t|  >  L,.  Then  sub¬ 
stitution  of  (98)  in  (97)  yields 

w  (t)  »  w.(t)  ,  W  (f)  *  W  (f)  . 

*  d  *  d  (100) 


That  is,  the  effective  weighting  and  window  are  equal  to  the  desired  behavior.  We 
have  “undone”  the  effects  of  bad  side  lobes  in  temporal  window  W,(f)  by  reshaping 
according  to  lag  weighting  w2(t)  in  (98).  (The  effect  on  the  variance  of  the  second- 
stage  spectral  estimate  62(f)  will  be  considered  later.) 

To  see  how  much  can  be  accomplished  by  this  approach,  some  attainable  ef¬ 
fective  windows  that  can  be  realized  via  lag  reshaping,  for  continuous  rectangular 
temporal  weighting,  are  given  in  figure  13  for  the  largest  possible  value  of  L2, 
namely,  L2  =  L,.  Superposed  on  the  window  Wf(f)  for  rectangular  temporal 
weighting  are  the  effective  windows  for  four  candidate  lag  reshapings,  for  L2  =  L,. 
These  are  the  narrowest  possible  effective  windows  for  a  given  L,.  The  first  one  in 
figure  13A  corresponds  to  an  effective  Hanning  weighting.  The  peak  side  lobe  is 
only  reduced  from  -13.3  dB  to  -13.7  dB,  and  the  asymptotic  decay  is  improved  to  9 
dB/octave  from  6  dB/octave.  The  main  lobe  width  is  only  slightly  broadened. 

Much  greater  improvements  in  side  lobe  behavior  are  possible  with  other  lag 
weightings,  and  are  illustrated  in  parts  (B)-(D)  of  figure  13.  They  illustrate, 
respectively,  peak  side  lobe  levels  and  decays  of:  -30.5  dE,  21  dB/octave;  -41.3  dB, 
15  dB/octave;  and  -46.7  dB,  9  dB/octave.  The  deeper  peak  side  lobe  is  realized  at 
the  expense  of  a  slower  asymptotic  decay.  They  all  have  about  the  same  mam  lobe 
width.  TheC5,  C3,  Cl  weightings  were  introduced  and  explained  in  (60). 
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8  dB/OCTAVE  DECAY 


9dB/OCTAVE  DECAY 


13A.  Reshaping  to  Achieve  Hanning  Weighting 


K<f) 


6  dB/OCTAVE  DECAY 


21  dB/OCTAVE  DECAY 


8  9 


13B.  Reshaping  to  Achieve  C5  Weighting 


Figure  13.  Some  Attainable  Effective  Windows  via  Lag  Reshaping 
for  Rectangular  Temporal  Weighting  and  L,  =  L, 
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Figure  13.  Some  Attainable  Effective  Windows  via  Lag  Reshaping  for 
Rectangular  Temporal  Weighting  and  L2  =  L,  (Cont’d) 
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Figure  13  illustrates  how  advantageous  the  reshaping  technique  can  be  in  terms  of 
peak  side  lobe  and  asymptotic  decay,  although  the  main  lobe  width  is  significantly 
increased.  In  fact,  the  peak  side  lobe  at  f  »  1.5/L,  for  the  rectangular  window  is 
really  not  suppressed,  so  much  as  it  is  smeared  out;  however,  the  other  peaks  of 
Wflf)  for  |f|  >  2/L,  are  indeed  significantly  reduced.  Thus  reduction  of  leakage  via 
lag  reshaping  is  a  very  effective  method,  provided  that  we  accept  the  nearest  side 
lobe  of  the  temporal  window;  this  conclusion  is  in  contrast  to  reference  20,  page  57. 
These  general  conclusions  on  lag  reshaping  hold  also  for  temporal  weightings  other 
than  rectangular,  although  the  exact  degree  of  improvement  will  be  different. 

If  L2  is  chosen  less  than  L,,  the  effective  windows  in  figure  13  are  simply 
broadened  according  to  the  ratio  Lj/L2.  The  peak  side  lobe  levels  and  asymptotic 
decay  are  unchanged,  but  the  main  lobe  width  is  increased.  Here  we  are  presuming 
L,  fixed  and  decreasing  L2. 

If  we  insist  that  the  combination  of  temporal  weighting  w,(t)  and  lag  reshaping 
w2(t)  in  (98)  have  effective  bandwidth  Be,  then  use  of  (52),  (99)-(100),  and  (57)-(58) 
yields 

%<«>  »>>  , 

Be  “  f.  2.  .  *  ~T  T7T  *  2L_  c { w , }  ’ 

J  dr  wg(i)  J dT  wd(t)  2  d 

(101) 


where  c{wd}  is  the  shape  factor  of  wd(t)  (see  table  1).  Thus 


BeL2  e2F{wd}  for  L2-LliT 


(102  A) 


the  limits  on  L,  in  (102A)  follow  from  (98)  and  (15). 

A  plot  of  the  interrelationship  between  L,  and  L2  (introduced  in  figure  5)  is  shown 
in  figure  14  for  the  case  of  lag  reshaping.  The  reason  that  the  plot  is  flat,  in  contrast 
to  figures  5  and  6,  is  that  the  shape  of  w2(t)  now  changes  as  L,  changes.  This 
behavior  is  discernible  from  (98),  since  the  denominator  varies  while  the  numerator 
remains  fixed  according  to  the  selection  of  wd(T)  and  its  associated  bandwidth- 
length  factor  (102A). 

If  the  maximum  segment  length,  L,,  is  specified  (as  for  example  when  the 
maximum  FFT  size  and  the  time-sampling  spacing  A,  are  fixed),  the  condition  under 
which  a  desired  effective  frequency  resolution,  Bd,  can  be  met  is  given  by  figure  14. 
Namely,  we  see  that 


B 

e 


“  2  c{V  L1 


(102B) 


Thus  if  desired  resolution  Bd  is  greater  than  or  equal  to  the  right-side  of  (102B), 
there  exists  a  choice  for  segment  length  L,  that  will  yield  the  desired  frequency 
resolution.  The  shape  factor  in  (102B)  depends  only  on  the  desired  weighting  wd(T). 
(See  (65)  and  the  accompanying  discussion  for  the  case  where  lag  reshaping  is  not 
employed.) 
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Variance  of  Spectral  Estimate 


Up  to  this  point,  we  have  presumed  nothing  about  the  process  x(t)  except  that  it 
be  (second-order)  stationary  during  the  observation  interval  T.  Now  we  make  a 
couple  of  assumptions  about  the  process  in  order  to  obtain  manageable  expressions 
for  the  variance  of  the  second-stage  spectral  estimate  62(f),  at  frequency  f.  Our  first 
assumption  is  that  the  true  spectrum  G  of  x(t)  varies  slowly  in  the  neighborhood  of 
the  frequency  of  interest,  f.  More  precisely,  from  (50)  and  (70),  we  obtain,  for  the 
mean  spectral  estimate, 


Av{G2(f)} 


J 


du  G(f  -  u) 


We(u) 


G(f)  / 


du 


We(u) 


G(f) 


(103) 


where  we  assume  that  spectrum  G  is  relatively  constant  in  the  frequency  band 
(f-B/2,  f+B/2);  i.e.,  the  only  region  where  effective  window  We  in  (103)  is 
substantially  nonzero  is  in  the  range  (-Bg/2,  Be/2). 

Our  second  assumption  is  that  x(t)  is  a  complex  Gaussian  process.  The  variance  of 
62(f)  is  developed  under  this  assumption  in  appendix  D,  culminating  in  the  exact 
result  in(D-13): 


Var(G2(f)}  -  ff  da  d8  G(a)  G(0)  |y(f  -  a,  f  -  B)|2  Qp(S(a  -  0))  , 

(104) 


where  window  convolution  function 

y(x,  y)  =  j  du  W2(u)  Wj(x  -  u)  W*(y  -  u) 
and  periodic  function 


(105) 


(106) 


The  variance  result  in  (104)  does  not  require  that  spectrum  G  vary  slowly  in  the 
neighbor  of  f;  the  result  utilizes  only  the  Gaussian  assumption  on  the  process  x(t). 
The  temporal  and  lag  windows  contribute  through  the  window  convolution  func¬ 
tion  y,  while  the  shift  S  and  number  of  pieces  P  appear  through  the  periodic  func¬ 
tion  Qp. 


When  the  assumption  regarding  a  slowly  varying  spectrum  G  in  the  neighborhood 
of  frequency  f  of  interest  is  also  invoked,  (104)  simplifies  to  forms  given  in  (D-20) 
and  (D-24);  the  latter  is  a  “weighting  domain”  version  of  the  variance: 


Var{G2(f)}  =  G2(f) 


i  P-1 

¥  £ 

p=l  -P 


w 2 (T) 


4>t(t,  pS) 


(107) 
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where 

VT-“>  5  fdc  "l  (*  *  H-1)  “l(l  ‘  »i(*  *  H2-)  »: 

■  ♦jCiT,  +w) 

is  a  third-order  correlation  of  temporal  weighting  w,(t).  The  form  (107)  is  very 
useful  if  43  can  be  evaluated  in  closed  form.  An  “ambiguity  domain”  version  of  the 
variance  is  given  by  (D-20). 


(108) 
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Quality  Ratio 

The  quality  ratio  for  spectral  estimation  was  defined  in  (4).  With  the  aid  of  (103) 
and  (107),  it  is  given  by 

Var{£2(f)} 

Q  Av2{£2(f)} 

"  F  p£.p  (*  *  ^)/dT  w2(t)  *3(t’  pS)  '  (109) 

Since  the  smallest  possible  value  of  Q  is  (TB^-1  (see  (11)  et  seq.),  the  normalized 
quality  ratio  is 

NQR  i  Q  •  TBe  •  ^  £  (l  -  M)  f  dT  w2(t)  ^(t,  pS)  . 

P*1'P  (110) 

This  quantity  can  never  be  smaller  than  unity. 

If  we  employ  (52)  and  the  normalizations  (67)  and  (69),  the  convenient  form 

F  t  (‘  -  Jf1)  Idl  ”2(t)  ♦s(t-PS) 

NQR'T  (,)*5(,)  on) 

for  the  normalized  quality  ratio  is  obtained.  We  are  interested  in  the  behavior  of  the 
normalized  quality  ratio  for  different  choices  of  P,  S,  w,(t),  and  w2(t).  The  con¬ 
straint  of  a  fixed  effective  bandwidth  Be  has  been  injected  into  the  normalized 
quality  ratio  via  the  use  of  (52)  in  (1 1 1).  The  quantities  +,  and  needed  in  (1 1 1)  are 
given  by  (44)  and  (108)  respectively. 

Before  we  embark  on  particular  cases,  some  general  observations  on  overlap 
(shift  S)  are  in  order.  For  a  minimum  normalized  quality  ratio  (minimum  variance) 
with  each  temporal  weighting  w,(t),  we  should  use  approximately  the  optimum 
overlap  as  derived  in  reference  9.  For  example,  Hanning  temporal  weighting  should 
be  employed  with  approximately  62  percent  overlap,  although  there  is  only  an  8 
percent  loss  in  stability  if  50  percent  overlap  is  used  for  convenience  (reference  9, 
tables  5  and  6).  There  is  no  point  in  considering  excessive  or  inadequate  overlap, 
since  this  leads  to  excessive  computational  effort  or  more  variance,  respectively. 
Inadequate  temporal  overlap  cannot  be  made  up,  in  terms  of  variance  reduction,  by 
any  amount  of  quadratic  smoothing.  This  can  be  seen  by  observing  that  poor  first- 
stage  correlation  estimates  ^(t)  are  merely  multiplied  by  lag  weighting  w2(t),  and 
are  not  improved  statistical!',  in  any  way  for  |t|  <  L2;  those  estimates  for  |t|  >  L2  are 
discarded  by  the  lag  weighting. 

Some  related  work  on  the  effects  of  windowing  on  stability  is  given  in  references 
21  and  27.  However,  the  present  report  is  more  thorough  and  detailed  in  its 
treatment  of  the  problem  and  the  inclusion  of  a  bandwidth  constraint. 
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Special  Cases  of  Generalized  Spectral  Analysis  Technique 

This  section  will  consider  several  special  cases  of  the  normalized  quality  ratio  and 
show  how  some  earlier  results  are  obtained  as  limiting  cases.  The  next  section  will 
treat  the  generalized  spectral  analysis  results. 


One  Piece,  P  » 1 

When  only  one  piece  is  used  in  the  first-stage  spectral  estimate  6,(0  in  (14),  we 
have  a  generalized  version  of  the  Blackman-Tukey  approach,  in  that  the  data  x(t) 
are  weighted  by  w,(t)  prior  to  computing  the  sample  correlation;  see  (16)  and  (13). 
Also,  we  allow  length  L,  <  T  (although  we  soon  show  that  the  best  L,  is  equal  to  T, 
the  available  record  length).  From  (111), 


NQRj 


T 


J dx  w*(t)  4»2  Ct) 
J dx  *2  (t)  4>j(t) 


(112) 


where  (using  (108)) 

*2(t)  S  *  fdt  wi(t  ♦  7)  wi(t  ‘  j)  “  /dt  "iCO  *fj(t  -  t) 


(113) 


is  the  correlation  of  the  squared  temporal  weight  function  wf(t). 


Now  if  L,  »  L2,  w2  is  much  narrower  than  or  +2.  In  that  case,  the  exact  shape 
of  w2  is  irrelevant,  and  (1 12),  (1 13),  (44),  figure  2,  and  Schwarz’s  inequality  yield 


♦,(0) 

NQR  -  T  -  *  T 

♦J(0) 


■—  for  L.  »  L 
L1  1 


(114) 


Equality  in  (1 14)  results  if  and  only  if  wf(t)  is  constant  for  |t|  <  L,/2;  furthermore, 
the  best  value  for  L,  is  then  its  largest  allowed  value  T  (see  (15)),  in  which  case  we 
have  Blackman-Tukey  processing  and 


NQR  (rectangular  w.)  ■  1  fo^  T  ■  L.  >>  L_ 

1  1  1  z  (115) 

This  result  agrees  with  reference  2,  section  B.8.  It  should  be  noted  that  L,  »  L2 
implies  BeL,  »  BeL2  >  .5/c{w2}  ~  1,  according  to  figure  5  and  table  1;  thus  stable 
estimates  result  in  this  case. 

Instead  of  rectangular  temporal  weighting,  consider  Hanning  weighting: 


WjCt) 


t|  <  Lj/2 


(116) 
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Then 


♦j(0)  »  /dt  wf(t)  -  1  ,  *2(0) 


and  (1 14)  yields 


J 


dt  Wl(t)  *  181 


1  (117) 


35  T 

NQRj  (Hanning  wj)  *  "jj  L~  for  L1  >>  L2 


(118) 


The  best  L,  is  again  T;  however,  the  minimum  value  of  the  normalized  quality  ratio 
is  then  35/18,  which  is  twice  the  value  in  (115)  for  rectangular  temporal  weighting. 
This  is  due  to  the  squandering  of  the  edges  of  the  available  data  record  by  the  small 
values  there  of  Hanning  temporal  weighting. 

Now  instead  of  assuming  L,  »  L2,  let  us  reconsider,  for  general  w,,  L,,  L2,  the 
normalized  quality  ratio  (1 12).  Since  w,(t)  is  zero  for  |t|  >  L,/2,  we  have  from  (44), 


bfx) 

♦  1(t)  -  J  dt  w  (t)  w  (t  -  t)  for  |  t  |  <  L  , 

a(-r) 

where 

a(x)  =  maxC-Lj/2,  -  L^/2  ♦  t)  j 

|  for  | t l  <  Lj  . 

b(r)  =  mindj/2,  1^2  +  x)  J 

Then  by  Schwarz’s  inequality,  (113),  and  (120), 


(119) 


(120) 


2  /-bCx)  ,  -  rb(x) 

(t)  <.J  dt  Wj(t)  w^t  •  x)J  dt  1 

a(x)  ■'a(x) 

(121) 

*  $2( t)  (Lj  -  | x | )  for  |t|  <  Lj  . 


Equality  is  realized  in  (121)  if  and  only  if  w,(t)  is  constant  for  |t|  <  L,/2;  that  is,  the 
best  temporal  weighting  for  maximum  stability  is  rectangular  when  P  =  1 .  This 
conclusion  holds  regardless  of  the  form  of  lag  weighting  w2(t)  or  the  relative  sizes  of 
L,  and  L2. 

As  an  example,  for  rectangular  temporal  weighting, 

«i(t)  *  Lj"1*  for  |t|  <  Lj/2  , 

♦  1(t)  =  1  -  |  x  | /Lj  for  |  x  |  <  Lj  , 

$2(t)  =  -  M/Lj)  for  | t |  <  L2  ,  (122) 
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and  (112)  yields  for  general  w2(t), 


(123) 


The  ratio  of  integrals  is  obviously  greater  than  1.  For  a  monotonically  decreasing 
lag  weighting  w2(t)  of  fixed  shape,  the  ratio  of  integrals  is  minimized  by  choosing  L, 
as  large  as  possible.  Since  the  leading  factor  also  has  the  same  behavior,  the  best 
value  for  the  normalized  quality  ratio  is 


We  cannot  give  numerical  values  to  this  ratio  of  integrals  until  we  select  a  lag 
weighting  w2(t)  and  determine  the  specific  value  of  L2(T);  see  figures  5  and  6.  But  if 
TBe  »  1,  which  is  the  usual  case  for  reasonably  good  spectral  estimates,  then 
L2«  L,  =  T  and 


NQR.(rect.  wj  ■  1  for  L,  *  T,  TB  »  1 

1  1  1  e  (125) 

This  result  holds  independently  of  the  exact  shape  of  the  lag  weighting  w2(t);  thus 
we  could  choose  w2(t)  such  that  the  effective  weighting  we(r)  in  (48)  has  good  side 
lobe  behavior,  as  discussed  in  an  earlier  section. 

No  Quadratic  Frequency-Smoothing 

No  quadratic  smoothing  corresponds  to 

W2(f)  =  6(f), 

w2(t)  =  1  for  all  t  .  (126) 

Thus  L2  =  00 ,  and  (109)  becomes 


Q 


■  I dT  VT,pS) 

.  M)  ♦Jcps)  , 


(127) 
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since 

Jdr  $3(t,w)  «  fit  J du  -  jOWjfu  +  t)  Wj(u  +  x  -  u) 

•  fdu  Wj(u)  WjCu  -  y)  4>j (p)  ■  (y)  (128) 

by  use  of  (108).  The  result  in  (127)  is  identical  to  reference  9,  equation  8,  when  we 
recall  definition  (109)  and  normalization  (67). 


Non-Overlapping  Segments 

Let  us  choose  time  shift  S  in  (13)  equal  to  the  segment  length  L,;  this  leads  to 
abutting  time  segments.  From  (15),  we  have 


PLj  -  T 


(129) 


where  we  have  chosen  to  use  up  all  of  the  available  data  length.  (This  is  different 
from  the  earlier  subsection  for  P  *  1  where  we  allowed  L,  <  T.)  The  general 
normalized  quality  ratio  in  (1 1 1)  reduces  to 


NQRp 


Jdx  w^Ct)  «P2  Ct) 

r  j  2  * 

Jdx  w2(x)  ^(x) 


(130) 


where  we  used  (129),  the  fact  that  w,(t)  is  of  length  L,,  and  (1 13). 

Once  again,  we  refer  to  bound  (121)  and  the  fact  that  equality  is  realized  only  for 
a  flat  weighting  w,(t).  Thus,  from  (130)  and  (122), 


NQRp(rect.  wp 


(131) 


for  any  (real  symmeti'  ;  tag  weighting  w2(x).  The  ratio  of  integrals  is  obviously 
always  greater  than  L.\.ty;  therefore,  for  a  monotonically  decreasing  lag  weighting 
w2(x)  of  fixed  shape,  values  of  L,  large  in  comparison  with  L2  are  preferred. 
However,  L,  »  L2  means  that 


TB  =  PL.  B  >>  PL,  B  > 
e  1  e  2  e  — 


P 

2cTwJT 


P 


(132) 


according  to  (129)  and  figure  5.  Thus  large  time-bandwidth  products,  TBe,  are 
required;  also  P  must  be  kept  small  enough  to  realize  L,  »  L2.  In  this  case,  we  have 


NQRp(rect.  w^)  »  1  for  TBg  >>1,  L^  >>  L^ 


(133) 
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regardless  of  lag  weighting  w2(t).  Qualitatively,  when  L,  »  L2,  the  edge  effects  of 
segmenting  the  data  x(t)  are  negligible,  since  only  a  small  fraction  of  the  utilized 
correlation  values  that  can  be  calculated  from  a  record  of  length  T  are  neglected 
when  using  segments  of  length  L, .  Stated  alternatively,  all  the  first-stage  correlation 
estimates  that  are  used,  namely  R,(r)  for  |t|  <  L2,  have  the  same  quality  (stability) 
whenL2«L,. 

An  example  of  the  exact  normalized  quality  ratio  for  Hanning  lag  weighting  and 
rectangular  temporal  weighting  is  afforded  by  substituting  the  equation 


w2(x)  ■  cos2  for  I  t  |  <  l2 

into  (131)  (see  figure  7A  for  the  effective  window): 


(134) 


(135) 


(136) 


Equation  (135)  is  plotted*  as  the  top  curve  in  figure  15A.  As  expected,  the 
normalized  quality  ratio  tends  to  1  as  L2/L,  tends  to  zero.  But  even  for  as  large  a 
value  as  L2/L,  =  .5,  the  normalized  quality  ratio  has  increased  only  by  12  percent. 
Thus  the  penalty  in  increased  variance,for  not  realizing  a  small  ratio  for  L2/L„is  not 
severe. 


Also  plotted  in  figure  15A  is  the  normalized  quality  ratio  for  the  three  lag 
weightings  introduced  in  (60)  et  seq.  They  all  lead  to  smaller  values  of  the 
normalized  quality  ratio,  for  the  same  value  of  L2/L,;  in  fact,  lag  weighting  C5 
incurs  only  a  7  percent  increase  in  variance  when  L2/L,  =  .5,  in  relation  to  the  ideal 
value  1.  The  reason  that  the  normalized  quality  ratio  is  lower  is  due  to  the  fact  that 
the  lag  weightings  drop  to  zero  faster  within  their  length  L2. 


Non-Overlapping  Segments;  Lag  Reshaping 

The  possibilities  of  lag  reshaping  have  been  discussed  earlier  with  regard  to  the 
mean  of  the  spectral  estimate  and  the  effective  window.  We  now  want  to  see  what 
effect  lag  reshaping  has  on  the  normalized  quality  ratio  in  (130).  Substitution  of  (98) 
in  (130)  yields 


NQRp(lag  reshaping)  * 


•The  quantity  TBe  is  not  involved  in  figure  15;  some  related  computational  considerations  are 
discussed  in  appendix  E. 


J dx  w2(r) 

Jdz  w2(t) 


♦  f(T) 


for 


L2  1  L1 


(137) 
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Figure  15.  Normalized  Quality-Ratio  for  Rectangular  Temporal 
Weighting  and  No  Overlap 
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The  special  case  of  rectangular  temporal  weighting  is  obtained  by  employing  (122) 
in  (137): 


NQRp(rect.  Wj,  lag  reshaping) 


dx  wJ(t)  ^ 
dr  w*(x) 


1 


for  L2  <_  Lj 
(138) 


The  division  by  +,(r)  in  reshaping  (98)  increases  the  variance  (for  a  specified  L2/L, 
and  for  wd(x)  *  w2(x))  above  that  in  (131),  because  we  are  more  heavily  weighting 
regions  where  the  denominator  in  (138)  is  smaller. 


Equation  (138)  is  plotted  in  figure  15B  for  desired  effective  weightings  of  Han¬ 
ning,  Cl,  C3,  and  C5.  Notice  that  the  abscissa  is  now  limited  to  L2/L,  <  1.  As 
expected,  the  normalized  quality  ratio  tends  to  1  as  L2/L,  tends  to  zero;  that  is,  we 
can  do  lag  reshaping  for  good  side  lobe  behavior  and  lose  little  in  terms  of  stability, 
provided  that  L,  is  chosen  sufficiently  larger  than  L2.  Of  course,  the  normalized 
quality  ratio  values  in  figure  13B  are  larger  than  those  in  figure  15 A,  for  the  same 
value  of  L2/L,.  As  an  example,  for  desired  effective  weighting  C,.  if  we  take 
L,  =  2L2,  the  increase  in  variance  over  the  ideal  value  is  only  9  percent.  Thus  lag 
reshaping  is  an  attractive  procedure  for  spectral  estimation;  recall  from  figure  14 
that  L2  is  set  by  the  specified  Be  and  the  shape  of  wd(T). 


50 


TR  6459 


General  Results  on  Stability 

We  now  return  to  the  general  normalized  quality  ratio  in  (111)  and  recall  con¬ 
straint  (15).  We  will  select  time  shift  S  according  to 

s  "  <lLi  •  (139) 

where  q  =  q{  w, }  is  a  fraction  specified  to  be  in  the  range  (0, 1]  and  is  dependent  on 
the  particular  temporal  weighting  w,(t)  employed.  The  observations  made  in  the 
paragraph  following  (111)  are  relevant  in  this  regard.  For  example,  with  no 
quadratic  frequency  smoothing  and  with  Hanning  temporal  weighting,  q  =  .39  (61 
percent  overlap)  is  virtually  optimum,  although  q  -  1/2  loses  only  8  percent  in 
variance-reduction  capability  (reference  9,  tables  4-7).  We  also  select  equality  in  (15) 
so  as  to  make  maximum  use  of  the  available  record  length,  i.e.,  minimum  variance 
of  the  spectral  estimate.  Then  we  have 


(P  >  1) 


(140) 


Thus  for  a  given  T  and  specified  shift  fraction  q,  L,  can  take  on  only  a  discrete  set 
of  values. 

Arbitrarily  large  values  of  P  are  not  allowed  in  (140),  because  this  would  result  in 
such  small  values  of  L,  that  the  bandwidth  constraint,  Be,  could  not  be  met.  From 
figure  5,  the  lower  bound  on  B^L,  limits 


2c{*.}  B  T  -  1 

P  -  1  ♦  - i - £ - 

max  q 


(141) 


(Actually,  P^  must  be  the  integer  part  of  the  right-hand  side.)  Thus  Pmax  depends 
on  the  temporal  weighting  w,(t)  directly  through  its  shape  factor  c{$,}  and  in¬ 
directly  via  the  selected  shift  fraction  q  =  q{w,}.  For  q  =  1,  no  overlap,  (141) 
reduces  to  (E-l). 

When  P  =  Pmtx,  L,  is  at  its  minimum  value,  and  L2  must  be  greater  than  T;  it  can 
be  » (see  figures  5  and  6A).  In  this  case,  there  is  no  quadratic  frequency  smoothing, 
and  we  have  the  situation  studied  in  detail  in  references  9  and  12,  and  mentioned 
earlier  in  (126)-(128).  At  the  other  extreme,  when  P  =  1,  we  have  Blackman-Tukey 
processing  with  the  generalization  that  the  temporal  weighting  need  not  be 
rectangular;  this  case  was  considered  in  the  previous  section.  The  range  of  values  of 
L2/L,  is  shown  in  tE-2)  and  (E-3)  to  be  very  wide  when  BeT  »  1,  which  is  a  usual 
practical  case. 

More  generally,  for  P  in  the  range  [1,  Pmax],  we  can  investigate  the  tradeoff 
between  the  amounts  of  temporal-  and  lag-weighting,  for  specified  resolution  Be 
and  for  specified  weighting  shapes  of  interest.  Below,  we  consider  the  two  cases  of 
rectangular  temporal  weighting  and  Hanning  temporal  weighting. 
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Rectangular  Temporal  Weighting 

It  was  shown  earlier  in  figure  15  that  rectangular  temporal  weighting  with  no 
overlap  results  in  small  values  for  the  normalized  quality  ratio,  whether  the  lag 
weighting  is  reshaped  or  not,  provided  that  L2  is  chosen  somewhat  smaller  than  L,. 
Now  the  question  arises  as  to  whether  one  should  use  any  overlap,  such  as  50 
percent,  with  rectangular  temporal  weighting. 

We  presume  BeT  »  1.  For  no  overlap,  the  estimate  of  first-stage  correlation 
R|(t)  at  t  ~  L,/2  has  only  half  the  degrees  of  freedom  as  the  estimate  at  x  =  0.  But 
with  50  percent  overlap,  the  degrees  of  freedom  for  estimation  at  x  ~  L,/2  are  about 
the  same  as  at  x  =  0.  This  is  why  50  percent  overlap  for  rectangular  temporal 
weighting  appears  attractive. 

However,  for  estimation  of  R,(x)  at  x  >  L,/2,  we  still  do  not  get  as  many  degrees 
of  freedom  as  for  Blackman-Tukey  processing,  because  some  data  points  never 
interact.  For  example,  although  at  x  ■  3L,/4  we  have  doubled  the  degrees  of 
freedom  by  using  50  percent  overlap,  we  still  have  only  about  half  of  the  number 
that  are  available  at  this  x  value  via  Blackman-Tukey  processing. 

In  order  to  ascertain  quantitatively  the  merit  of  overlapping  for  rectangular 
temporal  weighting,  we  have  evaluated  the  normalized  quality  ratio  (111)  for  lag 
reshaping  to  realize  a  desired  effective  weighting  equal  to  Cl  as  given  in  (60)  and 
(61) (reference  19,  figure  12).  That  is,  in  (111),  we  use 

w.(x) 

W2Ct)=CTFT  f0r  M<L2~L1  * 

1  (142) 

where  j,(x)  is  given  by  (122).  In  addition,  we  need  the  third-order  correlation  (108), 
which  is 


$3(t,u) 


M  ♦  I M  |  <  Lj 

(143) 


for  rectangular  temporal  weighting. 

For  the  two  cases  of  BeT  =  100andBeT  =  1000,  the  normalized  quality  ratio  has 
been  evaluated  for  q  =  1,  .75,  and  .5,  and  plotted  in  figure  16.  The  explanation  of 
the  behavior  of  the  curves  is  as  follows:  • 

q  =  1  No  Overlap 

If  P  =  1,  then  L2 «  L,  and  it  follows  that  for  all  |xj  <  L2,  R,(x)  is  estimated  with 
virtually  the  same  degrees  of  freedom  as  at  x  =  0,  where  we  have  the  maximum 
degrees  of  freedom  possible  to  estimate  R,(0).  As  P  increases  toward  Pmax,  then  L, 
tends  to  L2.  Now  R,(0)  is  still  estimated  with  the  full  degrees  of  freedom,  but  R,(x) 
for  x  #  0  is  estimated  with  fewer  degrees  of  freedom.  For  x  near  L2,  the  loss  in 
degrees  of  freedom  in  estimation  of  R,(x)  is  significant,  and  the  variance  increases. 
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q  =  .5  50  Percent  Overlap 

For  Pal,  there  is  no  overlap  and  conditions  are  identical  to  those  described 
above.  For  P  =  2,  the  sudden  increase  in  variance  can  be  explained  as  follows:  from 
(16),  the  first-stage  estimate  is 


A 

Rj(t) 


1 

2 


dt  y0(t)  y*(t 


(144) 


In  particular, 

Rj(0) 


(145) 


where  the  overall  weighting  O(t)  of  x*(t)  is  depicted  in  figure  17.  As  shown,  the 
overall  weighting  is  very  uneven,  causing  loss  in  stability.  As  P  increases  above  2, 
the  uneveness  of  the  overall  weighting  (for  q  =  .5)  occurs  only  towards  the  edges  of 
the  (0,  T)  interval,  yielding  a  decrease  of  variance,  since  more  data  points  tend  to  get 
the  same  overall  weighting,  insofar  as  their  effect  upon  the  estimation  of  R,(t)  is 
concerned.  However,  at  the  same  time,  the  effect  of  fewer  degrees  of  freedom  in 
estimation  of  Rj(t),  for  t  values  near  L2,  becomes  more  pronounced  as  P  increases 
and  L,  decreases;  this  is  true  even  for  the  50  percent  overlap  case  being  considered 
here.  Eventually,  this  effect  dominates,  and  the  variance  increases  with  P. 


Figure  17.  Overall  Weighting  of  x2(t)  for  q  =  .5,  P  =  2,  Rectangular  w,(t) 
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q  ■  .75  25  Percent  Overlap 

For  any  value  of  P  >  1,  the  overall  weighting  O(t)  is  very  jagged  (as  above),  and 
the  jaggedness  does  not  decrease  or  concentrate  near  the  edges  as  P  increases.  This  is 
true  for  any  overlap  greater  than  0  and  less  than  50  percent. 

In  summary,  for  rectangular  temporal  weighting,  the  smallest  values  for  the 
normalized  quality  ratio  are  realized  by  choosing  q  =  1,  no  overlap,  and  making  L, 
several  times  larger  than  L2.  This  conclusion  about  the  ratio  L2/L,  is  consistent  with 
those  reached  earlier. 

Hanning  Temporal  Weighting 


The  temporal  weighting  and  associated  correlation  for  this  case  are  given  by 
(116),  (C-9),  and  (C-10): 


Evaluation  of  third-order  correlation  +3(r,  \i)  in  (108)  is  rather  tedious;  the  end 
result  is  given  in  (F-l)  and  (F-2).  The  procedure  and  program  for  the  evaluation  of 
the  normalized  quality  ratio  is  given  in  appendix  F. 


The  normalized  quality  ratio  for  Hanning  lag  weighting  and  BeT  =  100  is  plotted 
in  figure  18  for  several  values  of  the  shift  fraction  q.  When  q  =  1,  no  overlap,  the 
small  values  of  the  Hanning  temporal  weighting  at  its  edges  cannot  be  compensated 
for,  by  any  choice  of  L2/L,,  and  the  variance  remains  at  approximately  twice  the 
ideal  value.  For  50  percent  overlap  of  the  Hanning  temporal  weighting,  q  =  1/2,  the 
situation  is  markedly  improved,  there  being  a  value,  L2/L,  =  .4,  at  which  the  excess 
variance  is  only  8  percent  above  ideal;  this  is  reminiscent  of  the  variance  ratio  for 
the  case  of  no  quadratic  smoothing  in  reference  9,  tables  5-7.  When  q  is  decreased  to 
3/8  or  1/4  (62.5  and  75  percent  overlap,  respectively),  virtually  the  ideal  variance 
reduction  can  be  achieved  by  choosing  L2  =  L,. 

In  figure  19,  the  shift  fraction  q  is  kept  at  3/8,  while  BeT  is  taken  at  both  100  and 
32.  The  smaller  value  of  BeT  leads  to  a  slightly  larger  loss  in  performance  because  of 
more  significant  edge  effects.  However,  even  so,  the  normalized  quality  ratio  does 
reach  a  very  desirable  level  only  4  percent  above  ideal  when  L2  ~  2L,. 


Figure  19.  Normalized  Quality-Ratio  for  Hanning  Temporal  Weighting 
and  Hanning  Lag  Weighting;  q  =  3/8 
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Summary 

The  possibilities  and  performance  of  a  generalized  spectral  analysis  technique 
employing  temporal  and  lag  weighting  have  been  investigated  in  terms  of  the  mean 
and  variance  of  the  spectral  estimate.  The  only  assumption  required  about  the 
process  under  analysis,  in  so  far  as  the  mean  is  concerned,  is  that  it  be  second-order 
stationary  over  the  observation  interval.  We  then  were  able  to  extract  a  simple 
expression  for  the  effective  window  involving  the  temporal  and  lag  windows. 

The  possibility  of  doing  lag  reshaping  to  achieve  desirable  effective  windows  was 
considered  in  detail  and  found  to  be  reasonable  for  a  wide  variety  of  windows  with 
good  side  lobe  behavior  and  decay  rates.  In  particular,  if  rectangular  temporal 
weighting  is  employed,  its  inherent  poor  side  lobe  structure  can  be  corrected  via 
proper  lag  weighting,  in  so  far  as  the  effective  window  is  concerned.  Strictly 
speaking,  the  closest  side  lobe  cannot  be  eliminated;  however,  all  the  other  side 
lobes  can  be  suppressed. 

The  effect  of  temporal  and  lag  weighting  on  the  variance  of  the  spectral  estimate 
was  evaluated  and  compared  with  the  ideal  value  for  large  BeT.  For  rectangular 
temporal  weighting,  it  was  found  that  small  values  of  L2/L,  and  no  overlap  led  to 
values  of  the  normalized  quality  ratio  virtually  equal  to  the  best 
attainable  by  any  spectral  analysis  technique.  The  comparison  is  made  under  the 
constraint  that  the  effective  frequency  resolution  Be  is  maintained  the  same  for  all 
techniques  under  consideration.  On  the  other  hand,  if  Hanning  temporal  weighting 
is  employed,  overlapping  must  be  used  for  maximum  variance  reduction  and  the 
length  ratio  L2/L,  ought  to  be  of  the  order  of  unity. 

Since  Hanning  temporal  weighting  requires  multiplication  of  each  and  every  data 
segment  (P  pieces)  and  significant  overlap  (~  50  percent),  whereas  rectangular 
temporal  weighting  requires  no  multiplication  and  no  overlap,  the  latter  approach  is 
a  strong  candidate  for  spectral  analysis,  particularly  since  excellent  effective 
windows  (low  side  lobes  and  rapid  decay)  and  virtually  ideal  variance  reduction  can 
be  achieved  by  proper  lag  weighting  and  choice  of  ratio  L2/L,.  Investigation  of 
other  cases  than  those  evaluated  here  can  be  achieved  by  appropriate  modification 
of  the  program  in  appendix  F.  A  major  analytical  task  will  be  the  evaluation  of  the 
third-order  correlation  (108),  if  temporal  weighting  w,(t)  is  taken  other  than  rec¬ 
tangular  or  Hanning. 


58 


TR  6459 


Appendix  A 

Comparison  of  Two  Bandwidth  Measures 

The  effective  bandwidth  of  narrowband  window  W0(f)  was  defined  in  (1)  as 

2 


f df  W0(f) 

B  .  14 - ?  ~ 

e  / af  w  (f) 

J  Q 

The  half-power  bandwidth,  Bh>  is  defined  as  the  solution  of 

1 

'  2  ’ 


(A-l) 


(A-2) 


where  it  is  assumed  that  window  W0(f)  is  real,  even  about  f0,  and  peaked  at  f0.  We 
let  Wevf)  -  W0(f + fg);  thus  We(f)  is  even  about  f = 0. 

The  inverse  Fourier  transform  of  lowpass  window  We(f)  is  called  the  weighting 


we(t)  *  J  df  exp(i2irfT)  WeCf) 


(A-3) 


We  consider  here  the  class  of  weightings  given  by 


w 

e 


(x) 


k>0 


cos 


^rki/L^  for  |  t  | 


<  L 


e 


(A-4) 


and  zero  otherwise,  where  {ak}  are  real  and  non-negative.  This  class  includes 
rectangular,  Hanning,  Hamming,  Blackman,  and  the  optimal  windows  of  Nuttall, 
reference  19.  The  Fourier  transform  of  (A-4)  yields  lowpass  window 


2L 

w  (f)=  2L  f  sin(2irL  i:) 
e  ^  e  e 


L  -(-,k 


k>0  (2Lef)2  -  k2 


2a0 

for 

o 

n 

c 

"Ini 

for 

n  ^  0 

(A-5) 


A  table  of  bandwidths  Be  and  Bh  and  their  ratio  is  given  below  for  the  window  in 
(A-5).  Although  these  bandwidths  vary  significantly  for  the  different  weightings, 
their  ratio  is  much  more  stable.  In  fact,  for  the  last  four  weightings  listed,  the  ratio 
is  constant  within  ±  1  percent.  The  weightings  listed  under  C5,  C3,  Cl  are  those 
given  in  reference  19,  figures  10, 1 1,  12;  the  notation  means 


A-l 


TR  6459 


CS :  continuous  fifth  derivative  of  weighting 
C3:  continuous  third  derivative  of  weighting 
Cl:  continuous  first  derivative  of  weighting 


(A-6) 


Table  A-l.  Bandwidths  for  Various  Weightings 


Weighting 

BeL. 

BhL« 

Be/Bh 

Rectangular 

0.5000 

0.6034 

0.8287 

Hanning 

1.3333 

1.0000 

1.3333 

Hamming 

1.2614 

0.9109 

1.3848 

Blackman 

1.6415 

1.1494 

1.4281 

C5 

2.2165 

1.5371 

1.4420 

C3 

2.0478 

1.4139 

1.4483 

Cl 

1.9544 

1.3444 

1.4537 
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Appendix  B 

Some  Lag  Weighting  and  Lag  Windowing  Considerations 
For  Discrete-Time  Processing 

It  is  convenient  here  to  define  an  equispaced  unit-impulse  train  by  the  notation 

6a(b)  -  £  «( b  -  na)  ,  (B_1} 

where  the  summation  on  n  extends  over  ±  00 . 

For  discrete  time  sampling  at  spacing  A,,  it  has  been  observed  in  (30)  that  6,(f) 
has  period  1/A,  in  f.  Therefore  lag  window  W2(f)  could  be  confined  to  |f|  <  (2A,)*1 
with  no  loss  in  generality,  in  so  far  as  its  effects  on  62(f)  by  means  of  (18)  are 
concerned.  In  fact,  for  a  general  lag  window  W2(f)  specified  arbitrarily,  the 
equivalent  band  limited  lag  window  is 


W2(f)  *  rect(Atf) 


w2(f) 


5  j  (f) 

j 


(B-2) 


where  we  will  utilize  definitions 

1  for  J x |  <  1/2 


rect(x) 


0  otherwise 


,  sinc(x) 


sin(TTx) 


TTX 


(B-3) 


and  where  denotes  convolution.  That  is,  W2(f)  is  aliased  into  the  band 
|f|  <  (2A,)*1,  and  only  this  band-limited  portion  is  retained  for  $2(f). 

A  way  to  demonstrate  this  mathematically  ir  .o  note  that  the  only  values  of  lag 
weighting  w2(t)  that  can  affect  &2(t)  are  the  samples 


w2(qAt)  *  J df  exp(i27tfqAt)  W2(f) 


(B-4) 


The  band-limited  lag  weighting  function  that  passes  through  all  these  specified 
values,  for  all  q,  is 


w2(t) 


£w2(q  At)  s inc  (•£— 

q  \  t 


(B-5) 


with  corresponding  Fourier  transform 


B-l 
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W2(f)  »  At  rect(Atf)  £  *2(qAt)  exp(-i2irfqAt)  (B-6) 

q 

■  At  rect(Atf)  £  exp(-i2irfqAt)  J du  exp(i2iruqAt)  W2(u) 

q 

*  rcct(Atf)  j  du  W2(u)  At  £  exp|^-i2ir(f  -  u)qAtJ 


rect(Atf)  j  du  W2(u)  5  j  (f  -  u) 


rect(Atf) 


l*2  (f)  •  6  j  (f) 

*7  -f 


(B-7) 


Relation  (B-4)  indicates  how  an  arbitrarily  specified  W2(f)  fixes  the  lag  weights  at 
the  sample  points.  For  the  reverse  problem,  where  sampled  lag  weights  {w2(qAt)} 
are  specified  for  all  q,  relation  (B-6)  gives  an  equivalent  lag  window,  in  particular 
the  band-limited  spectral  window,  which  results  in  the  same  estimates  R2(t)  and 
62(f).  Notice  that  W2(f)  is  not  uniquely  specified  by  samples  {w2(qAt)};  however, 
the  band-limited  W2(f),  which  realizes  weights  (w2(qA,)}  for  all  q,  is  unique  and  is 
given  by  (B-6). 

As  a  special  case  of  the  above,  consider  discrete  frequency  smoothing  with 
frequency  spacing  Af  =  (MA,)-1,  where  MA,  is  of  the  order  of2L,;  i.e.,  from  (38), 

Vf)  ■  ms;  £  K2n  *(f  -  a?;)  •  (B.8) 


where  we  set  W2n  =  0  for  |n|  >  M/2  without  loss  of  generality,  in  accordance  with 
the  observation  above  (B-2).  Then  lag  weights  (B-4),  given  now  by 


w2(qAt) 


MAt  ln|^/2 


W2n  exP^2xnq/M) 


(B-9) 


will  equivalently  accomplish  the  same  purpose.  This  last  relation  can  be 
accomplished  by  an  M-point  FFT,  where  W2  ±M/2  receive  the  same  complex 
exponential  weighting  in  (B-9). 

,'.t  should  be  noted  that  the  discrete  function  w2(qA,)  in  (B-9)  has  period  M  in  q; 
this  means  that  w2(qA,)  in  (B-9)  will  increase  in  magnitude  for  M/2  <  q  <  M.  If  k , 
is  nonzero  for  |q|  >  M/2,  this  lag  weighting  may  cause  a  problem.  One  guaranteed 
way  to  avoid  the  problem  is  to  choose  M/2  larger  than  the  nonzero  extent,  N,,  of 
^  lq*  Physically,  this  means  that  the  frequency  spacing  Af  =  (MA,)-1,  used  in 
frequency  smoothing  (B-8),  must  be  small  enough  so  as  not  to  miss  any  information 
in  6,(0-  Coarse  frequency  spacing  gives  spurious  results  for  6,(f).  (It  will  also  yield 
poor  effective  windows.) 
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Since  from  (24),  &  ,q  =  0  for  q  >  N,,  where  N,  is  the  number  of  time  samples  per 
segment,  only  a  finite  number  of  the  general  weights  {w2(qA,)}  in  (B-4)  affect  A2q. 
Thus  in  example  (B-8),  although  (B-9)  has  period  M  in  q,  only  the  values  for 
|q|  <  N,  are  relevant  to  the  effect  on  A2q;  more  generally,  the  values  yielded  by  (B-4) 
for  a  general  W2(f)  are  relevant  only  for  |q|  <  N,,  and  only  these  need  be  evaluated 
and  retained  if  we  choose  to  process  via  the  lag  domain. 


Now  let  us  consider  the  reverse  problem,  where  lag  weights  {w2(qAt)}  are 
specified  for  all  q,  and  we  wish  to  determine  some  allowable  lag  windows  W2(f)  that 
will  realize  the  same  estimates  ft2(r)  and  62(f),  but  which  take  advantage  of  the  fact 
that  only  w^qA,)  foi  jqj  <  N,  must  be  realized.  One  obvious  candidate  is  the  band- 
limited  lag  weighting  version 


w2(t) 


£  w  (qAj  sinc/rp  -  q) 

|q|<Nj  2  *  W  ! 


(B-10) 


notice  the  limitation  on  q  employed.  The  corresponding  lag  window  is 


W2Cf)  •  rect(Af)  £  w,Cq4.)  e'i2’£q4t 

2  *  '  |q|<N,  2  ' 

A  second  candidate  is 

82W  *  "2^  ®  6MA  ^ 


(B- 11) 

(B-12) 


provided  that  M  >  2N,-1;  this  provision  guarantees  non-overlap  of  the 
displacements  of  &2(t).  Then 

"2tf>  ■  #2<«  MS-  ■  E  «2n  S(f  -  MS-)  (B-13) 

z  MA  n  ' 

where 


W2n  5  MS-“2  (MS-)=i?r'ct(jf),  ?  “2(q4t>  exp(-i2,nq/M) 

t  \  t/  | q | <N  j 

(B-14) 

from  (B-ll).  Notice  that  (B-13)  has  the  form  of  discrete  frequency  smoothing  in 
(B-8);  (B-14)  gives  the  area  of  each  impulse  needed  in  (B-13).  Also  notice  from 
(B-14)  that  all  these  areas  are  zero  for  |n|  >  M/2;  thus  we  have  a  finite  sequence  to 
apply  in  the  frequency  domain,  which  is  equivalent  to  a  specified  finite  set  of  lag 
weights. 

Equations  (B-14)  and  (B-9)  are  complementary  to  each  other.  In  fact,  we  can 
derive  (B-9)  from  (B-14)  as  follows:  from  (B-14), 

W.  =  i  rectf.7')  V  v  exp(-  i  27inq/M)  , 

2n  M  W  q  (B-IS) 
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where 


w 

q 


w2(qAt)  for  |q|  <  Nj 
0  otherwise 


Then 

53  W.  exp(i2irnp/M)  «  £  exp(i2imp/M)  ~  rect(jj)51 

|n|S/2  2n  |n|S/2  M  VM'q  * 

■  51  "n  13  rectfS)  exp(i2irn(p 
q  ^  M  |n| <M/2  W 


where 


w 

P 


9 


Z(M)  = 
P 


1  for  p  *  0,  *_  M,  +_  2M,  . . . 
0  otherwise 


NowifM>2N,-l  (as  assumed  above  (B-13)),  then 

X  w7n  exp(i2irnp/M)  *  w_(qA  )  for  |q|  <  N, 
| n | <M/2  *  z  1 

This  is  (B-9)  for  |q|  <  N,t  which  is  the  only  range  that  affects  ft2q. 


(B-16) 

exp(-i2irnq/M) 

-  q)/M) 

(B-17) 

(B-18) 

(B-19) 


B4 


TR  6459 


Appendix  C 

Correlation  4i(t)  of  a  General  Class  of 
Temporal  Weightings 

The  class  of  temporal  weightings  of  interest  here  is  given  by  a  sum  of  complex 
exponentials: 

w,  (t)  ou  exp(i2nkt/L1)  for  |t|  <  L./2  , 

k  (C-l) 

and  zero  otherwise.  We  assume  that  the  coefficient  sequence  has  conjugate  sym¬ 
metry 


a  ^  *  o£  for  all  k  ; 

(C-2) 


then  wt(t)  is  real,  and  it  follows  that  the  (aperiodic)  correlation 
4>1  (t)  *  fit  wx(t)  wl(t  -  t) 


(C-3) 


is  also  real,  in  addition  to  being  even  about  r  =  0. 
Substitution  of  (C-l)  in  (C-3)  yields 


OjCt) 


£ 

km 


a,  a* 
k  m 


L./2 

/ 

t-Lj/2 


dt  expj^i2irkt/Lj  - 
for  0  <  x 


i2mn(t  -  t ) / L ^ J 
<  Lj  .  (C-4) 


This  can  be  evaluated  and  then  extended  to  t  <  0  by  the  use  of  the  even  character  of 
|,(t);  there  follows 


for  |  t  |  <  l1  ,  (C.5) 

and  zero  otherv  :se.  This  is  the  general  result  for  the  correlation  of  weighting  (C-l). 
We  now  specialize  (C-5)  to  the  case  of  real  symmetric  coefficients  in  (C-l): 


“k  real *  a-k  =  “k 


(C-6) 
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For  all  coefficients  zero  except  for  a0,  we  have 


WiCt)  *  °p 

for 

|t|  <  Lj/2  , 

(C-7) 

♦jCt)  *  (Lx  -  | T I ) 

2 

°0 

for  | t |  <  Lj 

(C-8) 

For  all  coefficients  zero  except  for  a0,  or,,  we  have 
v^Ct)  =  Op  ♦  2oj  cos^nt/Lj)  for 

|t|  <  Lj/2  , 

(C-9) 

^(t)  *  (Lj  -  j t  I )  [dp  +  cos(2iTT/L1)j 
L1 

♦  —  a^ap  -  ctj)  sin(2ir ] t  | /Lj)  for  |t|  <  Lj  .  (C  1Q) 

For  all  coefficients  zero  except  for  aQ,  a,,  o2,  we  have 
Wj(t)  =  Op  +  20^  C0s(2irt/L^)  ♦  cos(4irt/Lj)  for  |t  |  <  Lj/2  t 

(C-ll) 

(t)  =  (Lj  -  I T  I)  +  2a2  cos(27tt/L1)  +  2a2  cos (4ttt/L1)J 

+  6?  [2ol(6a0  '  3al  "  4ot2)  sinC^M/Lj) 

-  a2(6a0  -  16aj  +  3“2^  sin(4ir  |  x  |  /Lj )J 

for  | x )  <  Lj  .  (C-12) 

For  all  coefficients  zero  except  for  a0,  a,,  a2,  a3,  we  have 

W1  (t )  =  Op  +  2 Oj  cosUnt/Lj)  +  2a2  cos^nt/Lj)  +  2a3  cos^iit/Lj) 

for  1 1 1  <  Lj/2  ,  (C-13) 
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^(t)  *  (Lj  -  I T  I ) 

t  2  2  2  2  1 

aQ  ♦  20j  cos(2irx/L1)  +  20^  cos(4ttt/L1)  +  2a^  cosCSttt/Lj) 

I*,  r 

+  307  |Sal^12a0  "  6al  “  8a2  +  3a3^  sin(2?rlTl/Li) 

-  «2(30aQ  -  80Oi  +  lSa^  ♦  48aj)  sin(4u |t | /L ^ ) 


♦  oijC 20<Xq  -  45aj  ♦  -  lOct^)  sin(6ir  |t  l/lpl 


for  | t  <  Lj  . 


(C-14) 


This  last  case  includes  all  the  weightings  considered  in  reference  19,  with  the 
identification  of  coefficients  as 


ao  *  ao 


for  k  >  0  . 


(C-15) 


Then  we  can  express  the  temporal  weighting  as 

w,  (t)  ■  £  a.  cos(2irkt/L.)  for  |t|  <  L./2  . 

k>0 


(C-16) 


C-3/C-4 
Reverse  Blank 
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Appendix  D 

Derivation  of  Variance  of  Spectral  Estimate 


Our  starting  point  is  (14).  The  integral  on  t  is  a  Fourier  transform  of  the  product 
in  (13),  and  can  therefore  be  expressed  as  a  convolution: 


Yp(f)  -  X(f)  •  |  W,(f)  exp 


Wl< 


(-i2irf(-f  +  PS))J 

Jdu  X(f  -  u)  Wj(u)  exp^-i2iru^y-  +  psjj^  , 


(D-l) 


where  we  used  (46)  and  defined 

X(f)  *  fit  x(t)  exp(-i2irft) . 


(D-2) 


Although  the  relations  to  follow  could  be  derived  in  the  time  domain,  it  is  more 
convenient  to  develop  them  in  the  frequency  domain  because  of  the  frequent  and 
useful  occurrence  of  delta  functions. 

X(f)  is  complex  Gaussian  for  all  f,  since  (D-2)  is  a  linear  transformation  and  we 
have  assumed  x(t)  to  be  a  complex  Gaussian  process,  for  the  variance  calculation  to 
follow.  Furthermore  covariance 

AvfXffj)  X*(f2)}  *  JJi itj  dt2  x(tj)  x*(t2)  exp(-i2irf1t1  +  i2Trf2t2) 

*  ffdti  dt2  ^  «xp(-i2irf1tj  +  i27rf2t2) 

=  jj  du  dt2  R(u)  exp(-i2irf  j  (u  +  t2)  +  i2nf2t2) 


=  G(fj)  6(f1  -  f2) 


(D-3) 


upon  use  of  (43)  and  (45).  When  x(t)  is  a  single-sided  (analytic)  complex  process, 
there  then  follows  for  the  fourth-order  average,  which  will  be  needed  later 
(reference  23), 


*v{X(fj)  X*(f2)  X(f3)  X*  (f4)  } 

=  G(fj)  G(f3)[6(f1  -  f2)  5(f3  -  f4)  ♦  5(f1  -  f4)  6 (f 2  -  f3)] 

(D-4) 
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(When  x(t)  is  a  real  process,  there  would  be  a  third  term  in  addition.  However,  even 
then,  this  term  contributes  only  near  f  =  0;  see  reference  9,  equations  (A-4)  -  (A-8).) 


Upon  use  of  (D-l),  (14)  becomes* 

a  i  p-1  ft 

G^f)  «  £  £  JJ  du  dv  X(f  -  u)  X*(f  -  v) 

p«0  J  J 

•  Wj(u)  W*(v)  exp 


Then  the  average  of  the  product  of  the  first-stage  spectral  estimates  is 

P-1 

Av{Gj(fj)  Gj^)}  ■  £  ,  ////  du  dv  dp  dv 


X(f j  -  u)  X*(f1  -  v)  X(f2  -  ji)  X*(f2  -  v)  Wj (u)  Wj* (v)  W^p)  W*(v) 


exp(-i2Tr(u  -  v) 


♦  P.s^j  exp^-i2ir(p  -  v)^y-  ♦  qsjj  . 


(D-6) 


Reference  to  (D-4)  enables  us  to  express  the  fourth-order  average  as 

G(fi  -  u)  G(f2  -  u) 

’  [«  (u  -  v)  «(U  -  v)  ♦  6(fj  -  f2  ♦  v  -  q)  «(fj  -  f2  +  p  -  v)J  .  (D-7) 
Use  of  the  first  term  of  (D-7)  in  (D-6)  yields 

-3  ffdu  dp  G(fj  -  u)  G(f2  -  p)  |Wj(u) |2  |W1(y) | 2 

P  p,q=0 

=  J du  G (f j  -  u)  | WL (u) | 2  •  j dp  G(f 2  -  p)  |Wj(u) |2 

=  Av{Gj (f j) )  ’  Av {Gj (f 2) }  ,  (D_g) 


where  we  employed  (45)  in  the  last  line.  Moving  this  term  to  the  left  side  of  (D-6), 
and  using  the  second  term  of  (D-7),  we  obtain,  for  the  covariance  of  the  first-stage 
spectral  estimates, 


•For  more  generality,  we  allow  temporal  weight  wj  and  window  W ,  to  be  complex  in  this  appendix 


D-2 
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Cov{G  (f  ),  G.tfJ}  -  X  £  ffdu  dv  G(fj  -  u)  G(f j  -  V) 

1  c  P  p,q*0  *  J 

•  Wj(u)  W*(v)  WjCv  ♦  f2  -  fj)  W* (u  +  f2  -  fj)  exp(-i2ir(u  -  v)  (p  -  q)S) 
1  P-X  f  f 

*7  £  JJ  da  dB  G(a)  G(S)  -  a)  -  B) 

P  p,q-0 

•  Wj (f 2  "  8)  W*(f2  -  a)  exp(-i2ir(B  -  a)  (p  -  q)S) 


i  P'1  }  f 

=  -j  J  da  G(a)  WjCfj  -  a)  W*(f2  -  a)  exp(i2na(p  -  q)S) 

P  p  ,q=0  I  J 


P-1 

£ 


p*l-P 


da  G  (a)  Wx (fx  -  a)  W*(f2 


a)  exp(i2irapS) 


=//da  dB  G(a)G(B)  -  a)  W*(fj  -  B)  Wj (f 2  -  B)  W*(f2  -  a) 

2 

(D-9) 

Here  we  have  used  the  identities 

“T  £  exp(i2ir(p  -  q)u)  =  i  £  (l  -  M.)  exp(i2*pu) 

P  p  ,q=0  F  p=l-P  \  p  / 

tsin(TrPu)!2  _  _  ,  . 

P  sin(Tru)  "  '  (D-10) 

For  f,  =  f2,  (D-9)  checks  with  reference  9,  equation  (A-9);  more  generally,  it  is 
equation  D-2  of  reference  9.  We  observe  that  if  |f:  -  f,|  is  greater  than  the  effective 
bandwidth  of  temporal  window  W,,  (D-9)  will  be  small  since  W,(f,  -  a)  and 
W*(f2  -  a)  will  then  not  overlap  significantly  on  the  a-scale.  Also  notice  that 
spectrum  G  is  still  left  under  the  integral  sign;  i.e.,  there  are  no  assumptions  yet  on 
the  character  of  the  spectrum. 

We  are  now  prepared  to  consider  the  second-stage  spectral  estimate  <j,(f)  as  given 
by  (18): 

G2(f)  =  Jdf1  GjCfj)  W2(f  -  fx)  .  {D.n) 

Then 


sin(ffPS 
P  sin(ir 


(ttPSCo  -  B)) 


D-3 


TR6459 


G2(f)  -  Av{g2(£)J  -  /df^CjCfi)  -  Av{Gj (fj)jJ  W2(f  -  fj)  , 

(D-12) 

and  therefore  (recall  that  W2  is  real)  the  variance  of  the  second-stage  spectral 
estimate  is 

Varjc2(f)}  -  //dfj  ^ 2  ^{^(fj).  GjCV}  W2(f  "  fl}  W2(f  "  f2) 
-rt  (*-¥•)//  da  dB  G(a)  G(B)  exp(i2ir(a  -  6)pS)  |y(£  -  a,  f  -  S)  | 2 

-  f  j da  d8  G(a)  G(6)'  |y(f  -  a,  f  -  S)  |2  Qp(S(a  -  B))  ,  (EM3) 

where  we  used  (D-9)  and  (D-10),  interchanged  integrals,  and  defined  window 
convolution  function 

Y(x,y)  =  [  du  W  (u)  W  (x  -  u)  W*(y  -  u)  . 

J  2  1  1  (D-14) 


Relation  (D-13)  is  exact;  it  makes  no  presumption  about  the  relative  widths  of  the 
spectrum  G  and  the  windows  W,,  W2.  The  compact  expression  (D-13)  involves  the 
windows  Wp  W2  through  the  convolution  function  y,  and  involves  the  shift  S  and 
number  of  pieces  P  through  the  periodic  function  QP  defined  in  (D-10). 

The  window  convolution  function  y  in  (D-14)  realizes  its  peak  value  at  x  =  0,  y  =  0, 
and  is  rather  small  everywhere  else,  since  the  windows  are  virtually  unimodal  and 
rather  narrow.  In  fact,  a  special  case  is  the  diagonal  slice 

Y(x,x)  *  J du  W2(u)|W1(x  -  u)|2  *  | Wj (x) | 2  •  W2(x)  =  Wp(x)  , 

(D- 1 5) 

by  reference  to  (51).  Generally,  y(x,  y)  is  substantially  nonzero  only  in  the  region 
Be,  Be  at  the  origin  of  x,  y  space. 

We  now  employ  the  assumption  discussed  in  connection  with  approximation 
(103),  namely,  that  spectrum  G  is  relatively  constant  in  the  band  of  width  Be  about 
the  frequency  f  of  interest.  Then  the  major  contribution  to  the  variance,  (D-13), 
comes  from  the  region  near  a  =  (i  =  f  in  a,  /?  space.  There  follows  the  approximation 
for  the  variance  of  the  second  -stage  spectral  estimate, 
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Var 


G2(f)  j 


•  j  j  da  df3  exp(i2ir(a  -  3)pS)|y(f  -  a,  f  -  3)  |2 


-  G2(£)  i  £  (l  -  40  //dx  dy  exp(i2w(y  -  x)PS)|y(x,  y)|2 
p*l-P  '  ' 

■  G2(f)  f fdx  dy  | y(x,  y)|2  Qp(S(y  -  x))  ,  (D-16) 


where  we  used  (D-10)  again. 

We  now  simplify  the  double  integral  in  (D-16);  from  the  second  line  of  (D-16)  and 
from  (D-14), 

D  i//dx  dy  exp(i2ir(y  -  x)pS)  J  j  du  W2(u)  Wj(x  -  u)  W*(y  -  u)  |2 

=  J  j  du  dv  W2(u)  w2(v)|/ dx  exp(-i2irxpS)  Wj(x  -  u)  W*  (x  -  v)|2 

*  // du  dv  W2(u)  W2(v)  |Xl(pS,  u  -  v)  |2  f  (D-17) 


where  the  complex  ambiguity  function  of  the  temporal  weight  and 
defined  as 

XjCt  .  v)  *  f  df  exp(i2nfT)  W^f  ♦  y)  W*(f  -  y) 

*  J  dt  exp(-i2wvt)  W^t  +  y^  W*^t  -  y^  . 


window  is 


(D-18) 


Now  let  v  =  u  -  v  in  (D-17),  and  obtain  a  single  integral  for  D: 

D  *  j  dv | Xj (pS,  v)|2  J du  W2(u)  W2(u  -  v) 

=  /  dv  I  Xj  (pS ,  v)i2  x2(°,  v)  , 


in  terms  of  the  ambiguity  function  x2  of  the  lag  window  and  weight.  Substitution  of 
(D-19)  in  the  second  line  of  (D-16)  yields  for  the  variance, 


Var(G2(f)}  =  G2(f)  I  £^1  -  4)  /dv I 


XjCpS,  v) 


x2(0, 


v)  • 
(D-20) 


An  alternative,  and  perhaps  more  useful  form,  to  (D-20)  is  attained  as  follows; 
the  integral  in  (D-20)  is  expressible  (by  use  of  the  definition  (D-18)  of  the  ambiguity 
function)  as 


D-5 


(D-21) 


Jdv|xj(pS,  v)  |  2  J  dt  exp(-i2irvt)  w2(t) 

*  J  dt  w^ft)  J  dv  exp(-i2irvt)  |  Xj(pS,  v) 


Now  we  have  the  general  result  that 


/ dv  exp(-i2irvt)  Ixj^Ct,  v)|2 

=  J  dv  exp(-i2irvt)  j j  dt  ^  dt2  exp(-i2irv  (t^  -  t2)) 

*  wl(*l  +  j)  Wl^  "  l)  +  l)  W1  ^2  '  l) 
dtl  dt2  "l(*l  *  l)  “jfl  '  l)  ”i(*2  *  ?)  “l(£2  '  l)  SCt  *  ll  - 
■  f dt  1  “i(ii  *  7) »;('!  ■  1)  “if  *  'x  *  t)  “if  *  'i  ■  t) 

/ d-  “if  *  H^)  "if  *  IV-L)  “i(u  *  H1)  “if  *  ^V-1) 

-  *3(t,  t)  .  (D-22) 


This  is  a  third-order  correlation  of  temporal  weighting  w,;  see  the  footnote  to  (9). 
Thus  (D-21)  becomes 


J 


dt  w2(t)  4>3  (t,  pS) 


(D-23) 


and  the  variance  in  (D-20)  becomes 

P“  1  '  \ 

Var{G- (f) }  =  G?(f)  J  ^  (l  -  M)  (dt  J(t)  4,  (t,  pS) 

P  p=l-P  V  P  2  3  (D-24) 

This  “weighting-domain”  version  of  the  variance  is  very  useful  if  third-order 
correlation  can  be  evaluated  in  closed  form. 
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Appendix  E 

Computational  Considerations  for  Non-Overlapping  Segments 


The  curves  in  figure  15  for  non-overlapping  time  segments  are  drawn  over  a 
continuum  of  values  of  L2/L,.  However,  if  we  were  given  a  value  of  TBe,  all  values 
of  L2/L,  may  not  be  allowed.  To  develop  this  point,  suppose  that  we  pick  an  integer 
value  for  the  number  of  pieces,  P,  and  solve  for  L,Be  =  TBe/P  according  to  (129). 
From  figures  5  and  6,  this  dictates  the  value  for  L2Be,  and  hence  a  discrete  value  for 
L2/L|  is  specified  for  each  value  of  P.  The  number  of  pieces,  P,  can  range  from  1  to 


pm«,  where 


BeL1(rain) 


Be  T  1 

•  P«« 


2c{ }  B  T 
i  e 


(E-l) 


from  figure  5.  For  BeT  »  1 ,  we  have,  for 


P  .  t.  BeLj  .  Be  T.  BeL2  -1^-T  .  4  •  gtlAt  -T  «  1 

2  l  2  e 


(E-2) 


and  for 


P  *  P  ,  BL, 
max’  e  1 


2cT^T  ’ 


B  L,  >  B  T,  j-z.  2c{*  }B  T  »  1 
e  2  —  e  Lj  l  e 


(E-3) 


Thus  a  very  wide  range  of  discrete  values  of  L2/L,  is  allowed  when  BeT  »  1 . 

The  problem  with  this  approach  is  that  when  BeL,  is  calculated,  BeL2  must  be 
solved  for  from  the  integral  relation  (52)  (or  approxir  ttely  from  figure  6).  This 
tedious  procedure  can  be  circumvented  by  specifying  L2/L,  instead;  if  desired,  we 
could  then  use  (52)  to  determine  BeL,,  and  solve  for  P  =  BeT/(BeL,).  However,  P 
will  not  necessarily  turn  out  to  be  an  integer  for  a  given  fixed  T;  thus  only  a  set  of 
discrete  values  of  L2/L,  are  strictly  legal.  But  if  NQRpin  (130)  does  not  vary 
radically  with  L2/L,,  this  is  not  a  significant  limitation.  And  since  it  is  simpler,  we 
adopt  it.  In  Appendix  F,  wc cannot  avoid  the  calculation  of  BeL2  from  a  given  BeL,. 


fc-l/t-2 
Reverse  Blank 
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Appendix  F 

Computational  Considerations  for  Overlapping  With 
Hanning  Temporal  Weighting 


The  temporal  weighting  is  given  by  (146).  Substitution  in  (108)  and  evaluation  of 
the  integral  yields 

4 


♦j(t-  u)  * 


where 


VQ(ir  -  a  -  6)  ♦  £  (-l)k-1  V  sin(k(a  ♦  8)) 


k=l 


for  a  +  B  i  it 


«  ■  r"M  .  6  *  -f-|  U  | 

L1  L1 


C  =  cos(a)  *  cos/-?— \  ,  C  *  cos(6)  *  cos(—] 
a  ILJ  8  \1) 


(F-l) 


3  2  2  2  2  4  4 

V0  ’  ?  -  Ca  *  C8  +  4Ca  C8  +  Ca  +  C8  * 

V1  ■  Ca  CS(4Ca  *  4CS  -  *)  ' 

V2--7*t(C«*Cb)*C»C«  • 


VKV  V4  ' 


(F-2) 


The  procedure  for  the  evaluation  of  the  normalized  quality  ratio  follows.  We 
specify  a  value  for  BeT  and  select  a  temporal  weighting  w,(t)  and  a  lag  weighting 
w2(t).  We  then  evaluate  shape  factor  c{$,}  from  (64)  or  table  2,  and  select  a  shift 
fraction  q  =  q  { w, }  according  to  (139).  We  then  solve  (141)  for  Pmax,  and  allow  P  to 
take  integer  values  in  the  range  1  <  P  <  Pmslx.  BeL,  can  then  be  evaluated  from 
(140)  as 


B  T 

D  I  _ _ ® _ 

el  "  1  ♦  (P  -  l)q 


(F-3) 


for  each  integer  value  of  P.  We  then  solve  (52)  in  the  form 

fdT  w 2  (t)  *2  (t)  =  / dx  w^L  x)  *|(L  x) 
e  1  l  1 


(F-4) 


for  the  ratio  L2/L|.  w2(L,x)  is  a  function  of  L2/L,,  while  ^(L  ,x)  is  independent  of 
L,  and  L2.  Next  we  compute  quality  ratio  (109)  in  the  form 


F-l 


TR  6459 


Q  -  J  Ep  (!  ‘  -^')/dx  w2^L1x^  L1*3(L1x*  pq  LP  (F-5) 

where  we  let  t  *  L,x  and  used  (139).  The  quantity  w2(L,x)  is  a  function  of  only 
Lj/L,,  while  the  remaining  quantity  in  the  integrand  of  (F-5)  is  independent  of  L, 
and  L2.  Finally,  we  multiply  (F-5)  by  BeT  according  to  (110)  in  order  to  determine 
the  normalized  quality  ratio. 

To  reduce  computation  time,  we  take  advantage  of  various  properties  of  the 
functions  involved.  First,  since  43(t,  p)  is  even  in  r  and  \i  (see  (108)),  and  w2  is  even, 
we  express  (F-5)  as 


Q  *  F  €k(*  "  j)  fQ  **  w2(Lix)  L1*3(L1x’  qk  Ll5 

where 

1/2  for  k  -  0 
1  for  k  >.  1 


(F-6) 


(F-7) 


Also,  from  (F-l)  and  (F-2),  we  have  normalized  form 


{v‘ 


a,  Ljb)  *  £  Jv0  n(l  -  a  -  b)  ♦  £  (-l)k_1  Vk  sin[k,T(a  +  b)] 


) 


for  0  <.  a,  0  <_  b,  a  ♦  b  <.  1  , 


(F-8) 


where  now 


C  *  cos(ira),  Cn  =  cos(irb)  , 
a  p 

(F-9) 


and  { Vk};J  are  still  as  given  in  (F-2). 

Since  L,+3(L,x,  pqLj)  in  (F-6)  is  zero  if  x  +  qk  >  1,  we  can  limit  the  sum  on  k  in 
(F-6)  to  km  =  min  (P  -  1,  1/q),  and  we  need  to  evaluate  the  integral  on  x  in  (F-6) 
only  up  to  Xm  =  min  (L2/L,,  1  -  qk).  The  number  of  x  intervals  needed  in  (F-6)  is 
about  16  with  the  Trapezoidal  rule  for  integration.  These  features  are  incorporated 
in  the  program  listing  below. 
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1  !  NUSC  TECHNICAL  REPORT  6459,  29  MAY  1981,  A.  H.  NUTT  ALL 
10  Bet “100  !  Be  T 

26  Cl*l/6+35/<48*PIA2)  !  HANNING  7EH»0RAL-UE I GHT I NG  U  1 ;  TABLE  2 

30  <31*3/8  !  62.5-:  OVERLAP  FOR  Ml;  0< Q 1  < * l 

40  Pm*l+<2*Cl*Bet-l >/<31  !  Pm*x;  EQ .  141 

50  PRINT  "Be  T  *";Bet,"q  ■  q<wl>  *";Q1 

60  PRINT 

70  COM  T1,T2,T3,T4 

80  T2*2*PI 

90  T3-1/3 

100  T4-1/T2 

110  FOR  P-1  TO  Pm 

120  Bel l*Bet/<l+<P-l)*Ql)  !  Be  LI;  EQ.  140 

130  L2 1 1 -FNL21 1 < Be  1 1 , Cl >  !  L2/L1J  SOLUTION  OF  EQ.  52 

140  T5-.5#PI/L21  1 

150  Km*MIN<P-l, 1/Q1) 

160  S*.5*FNInt<0,L21 1,T5> 

170  FOR  K-l  TO  Km 

180  S-S-K  l-K/P>*FNInt <Q1*K,L21  1 ,  T5> 

190  NEXT  K 

200  Q-4/P*S 

210  PRINT  P.L21 1 , Q*Bet 

220  NEXT  P 

230  END 

240  ! 

250  DEF  FNL2 1 1 <  Be  1 1 , C 1 >  !  SOLVE  EQ.  52  FOR  L2  LI 

260  Ep**Bel 1-. 5/Cl 

270  Xl-4/<3*Bel  1  > 

280  IF  Bel  1 < 1 0  THEN  XI* . 603246/SQRC Eps >-Eps/ <6+ 1 2*Ep* ) 

290  X2-X 1*1. 037 

300  Fi«FNF<Xl, Bel  1) 

310  F2*FNF  <X2, Be  1 1 ) 

320  IF  ABS<F2-F1X  IE-6  THEN  330 
330  T  *X2 

340  X2*<F2*X1-F1*X2)/<F2-F1) 

350  Xl-T 

360  F 1 -F2 

370  GOTO  310 

380  RETURN  X2 

390  FNEND 

400  ! 
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410  DEF  FNF<L21 1 , B*1 1 J  !  RIGHT  SIDE  -  LEFT  SIDE  OF  EQ.  F-4 

420  COM  T 1 

430  Xm-MIN<  1,L21 1) 

440  Tl*.  5#PI/'L21  1 
4S0  N-16 

460  D*1*Xm/N 
470  F".  5 

480  FOR  K* 1  TO  N-l 
490  F«F+FNG<K*D*1 > 

500  NEXT  K 

510  RETURN  2*D*1 *F-1/B*I 1 
520  FNEND 
530  I 

540  DEF  FNG<X>  !  <U2<L1*X > *Ph i  1  <L  1  *X>  > '-2 

550  COM  T1,T2,T3,T4 
560  P«T2»X 

570  G«<1-X)*<2+C0S<P))*T3+T4*SIN(.P) 

580  W2*C0S<Tl*X>/'2  !  HANNING  lAG-WEIGHTING  W2 

590  RETURN  <W2*C)A2 
600  FNEND 
610  ! 

620  DEF  FNInt  <Qk , L21 1 , T5>  !  INTEGRAL  OF  EQ.  F-5 

630  .  Xm«MIN<L21 1, i-Qk) 

640  D«l«Xm/'16 

650  S». 5*FNPhi 3<  0, Qk ) 

660  FOR  J-l  TO  15 

670  X»Dt 1  * J 

680  S«S+COS<T5*X> A4*FNPhi  3<X,  Qk  > 

690  NEXT  J 
700  RETURN  D«1*S 
710  FNEND 
720  ! 

730  DEF  FNPhi 3< A, B)  !  L 1 *PH I3<L1*A,L1*B>  for  A>  =  0,  B>«0;  EQ.  F-3 
740  IF  A  +  B >■ 1  THEN  RETURN  0 
750  Ab»PI*<A+B> 

760  C*-COS<PI*A> 

770  Cb»COS<PI#B> 

780  C*2»C*A2 

790  Cb2*CbA2 
800  S«C*2*Cb2 

810  V0». 375-S+4*Ca2*Cb2+Ca2A2+Cb2A2 

820  Vl«C**Cb*<4*S-l > 

830  V2— .25  +  .5*S+C*2*Cb2 

840  V3«C**Cb/3 

850  S»Vl*SIN<Ab>-V2*SIN<2*Ab>+V3*SIN<3*Ab>-.03125*SIti'  4*Ab, 

860  LI phi 3-4/<9*PI  >*<V0*<PI-Ab>+S> 

870  RETURN  Llphi3 
880  FNEND 
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Two-Channel  Linear- 
Predictive  Spectral 
Analysis;  Program 
For  the  HP  9845 
Desk  Calculator 


A.  H.  Nuttall 

ABSTRACT 

A  program  for  llnear-p  edictlve  spectral  analysis  of  two  channels  of 
data,  including  estimation  of  the  auto-spectra,  cross-spectrum, 
magnitude-squared-coherence,  and  the  argument  of  the  coherence, 
is  presented  in  BASIC  for  the  HP  9845  desk  calculator.  Timing  results 
for  the  major  subroutines  are  included,  and  their  dependence  on  the 
fundamental  parameters  of  the  data,  filter,  and  desired  spectral 
resolution  is  indicated.  These  techniques  and  program  for  spectral 
analysis  are  very  appropriate  for  short  data  segments.  In  particular,  a 
positive-definite  spectral  matrix  estimate  is  guaranteed. 

Applications  to  examples  including  a  strong  tonal  interference  in  one 
channel  are  made,  and  a  possible  shortcoming  of  the  technique  is 
pointed  out.  A  suggested  remedy  is  proposed  and  a  philosophy  for 
multichannel  spectral  analysis  is  suggested  for  further  con¬ 
sideration. 


Approved  for  public  release;  distribution  unlimited. 
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Two*Channel  Linear-Predictive  Spectral  Analysis; 

Program  for  the  HP  9845  Desk  Calculator 

Introduction 

Spectral  analysis  of  short  data  segments  by  the  standard  FFT  procedure  is  not  a 
viable  approach;  unstable  and/or  coarse  estimates  of  the  spectra  result.  An  at¬ 
tractive  technique  in  this  case  is  linear-predictive  spectral  analysis,  both  for  the 
single-channel  as  well  as  the  multiple-channel  cases.  See  references  1-9,  particularly 
references  7-9  which  derive  and  give  Fortran  programs  for  a  multiple-channel 
linear-predictive  spectral  analysis  technique  that  is  a  generalization  of  Burg’s 
technique  for  the  single-channel  case  (reference  1). 

The  purpose  of  this  report  is  twofold:  first,  we  translate  the  Fortran  program  in 
reference  9  into  Basic  for  use  on  the  Hewlett-Packard  HP  9845  Desk  Calculator, 
and  in  the  process,  also  make  some  minor  improvements  and  modifications  to  the 
format  and  printout  statements.  We  also  limit  consideration  to  the  two-channel  case 
and  thereby  take  advantage  of  some  simplifications  in  computing  possible  for  this 
special  case.  Second,  we  apply  the  program  to  a  pair  of  stationary  processes,  one  of 
which  has  pure  tones  that  are  not  present  in  the  other  process.  In  this  manner,  we 
point  out  a  possibly  deleterious  effect  on  the  auto-spectral  estimates  and  the 
coherence  estimate,  and  indicate  a  method  for  circumventing  some  of  the  difficulty. 
As  a  byproduct,  a  philosophy  for  multichannel  spectral  analysis  is  suggested  for 
further  consideration. 
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Use  of  Program  For  Spectral  Analysis 

In  appendix  A,  the  listing  for  the  two-channel  iinear-predictive  spectral  analysis 
technique  is  presented.  Inputs  required  of  the  user  are  the  following: 

N  Number  of  data  points  in  each  process 
Pmax  Maximum  order  of  predictive  filter  to  be  considered 
Nfft  Size  of  FFT  to  be  used  in  spectral  computation.  (1) 

In  addition  to  these  integer  inputs,  the  user  must  modify  the  subroutine 
SUB  Data  (N,X(*))  to  accommodate  and  read  in  his  particular  two-channel  data. 
All  data  are  presumed  real. 

The  program  computes  the  (sample)  means  of  each  of  the  two  processes  and 
subtracts  the  means  from  the  data.  (Some  possible  ramifications  of  this  procedure 
are  considered  in  reference  6,  appendix  B;  in  addition,  the  effect  of  choosing  too 
small  an  FFT  size,  Nfft,  is  discussed  in  reference  6.)  Next,  the  covariance  matrix  (at 
zero  delay)  of  the  input  data  is  computed,  and  the  Akaike  Information  Criterion 
(AIC,  reference  8,  pages  42-44)  is  evaluated  and  used  to  select  the  integer 

Pbest  Best  order  of  predictive  filter  to  use.  (2) 

The  forward  and  backward  partial  correlation  coefficients  (references  7-9)  are 
evaluated  through  order  Pmax,  as  well  as  the  forward  predictive  filter  coefficients 
for  Pbest.  The  normalized  correlation  matrices  are  computed  through  Pmax  (ex¬ 
trapolated  values  beyond  Pbest)  and  the  spectral  density  matrix  is  computed  (via  an 
FFT)  from  zero  to  Nyquist  frequency,  fN  =  (2A)*‘,  where  A  is  the  time-sampling 
increment  of  the  processes.  A  partial  check  on  the  adequacy  of  the  FFT  size,  Nfft,  is 
afforded  by  a  printout  of  the  areas  under  the  spectral  estimates  and  comparison 
with  the  (sample)  covariances  of  the  input  data.  Finally,  the  inverse  FFT  of  the 
spectral  estimate  gives  the  aliased  normalized  correlation  matrices;  the  motivation 
and  equations  for  this  approach  are  given  in  reference  9. 

A  sample  printout  for  a  short  data  sequence  (20  data  points  in  each  process)  is 
given  after  the  program  listing  in  appendix  A,  as  a  test  or  check  case  on  a  user- 
written  program.  Also,  plots  of  the  corresponding  auto-spectral  estimates  and  the 
coherence  estimates  are  given  there  for  completeness,  although  this  example  has  no 
real  physical  significance. 

Timing  Results 

Execution  times  for  the  five  major  subroutines, 

Pcc  Partial  correlation  coefficients 

Pfc  Predictive  filter  coefficients 

Peftf  Predictive  error  filter  transfer  function 

Sdm  Spectral  density  matrix 

Acm  Aliased  correlation  matrices,  (3) 
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arc  given  in  tables  1-5  below,  for  the  HP  9845B  Desk  Calculator  equipped  with  the 
Fast  Processor  Upgrade  Kit.  Only  those  variables  utilized  in  each  subroutine  are 
considered  in  these  tables,  since  execution  time  is  independent  of  the  other 
variables;  for  example,  the  execution  time  of  subroutine  Pcc  does  not  depend  on 
Pbest. 

Table  1.  Execution  Times  for  Subroutine  Pcc 


N 

Pmax 

Seconds 

20 

6 

1.9 

50 

10 

5.4 

100 

5 

4.7 

100 

10 

9.5 

100 

15 

14.1 

1000 

47 

404.2 

Table  2.  Execution  Times  for  Subroutine  Pfc 


Pmax 

Pbest 

Seconds 

5 

1 

.09 

10 

1 

.15 

6 

4 

.24 

15 

5 

.62 

15 

11 

1.41 

47 

12 

4.06 

Table  3.  Execution  Times  for  Subroutine  Peftf 


Pbest 

Nfft 

Seconds 

4 

256 

17.5 

11 

256 

17.8 

1 

512 

32.0 

5 

512 

32.8 

1 

1024 

63.9 

11 

1024 

66.6 

Table  4.  Execution  Times  for  Subroutine  Sdm 


Nfft  Seconds 

256  8.9 

512  17.7 

1024  35.3 


Table  5.  Execution  Times  for  Subroutine  Acm 


Nfft 

Seconds 

256 

9.8 

512 

18.6 

1024 

37.7 

From  these  tables,  we  are  able  to  extract  the  following  fairly  accurate  rules  of 
thumb:  the  execution  time  of 

Pcc  is  linearly  dependent  on  N  and  Pmax 
Pfc  is  linearly  dependent  on  Pmax  and  Pbest 

Peftf  is  linearly  dependent  on  Nfft,  but  is  essentially  independent  of  Pbest 
Sdm  is  linearly  dependent  on  Nfft 

Acm  is  linearly  dependent  on  Nfft.  (4) 

These  rules  allow  extrapolation  to  other  cases  of  interest  to  the  user.  The  execution 
times  of  the  FFT  itself  are  given  in  table  6. 

Table  6.  Execution  Times  for  Subroutine  FftlO 


Nfft 

Seconds 

128 

2.6 

256 

4.5 

512 

8.4 

1024 

17.1 

If  the  user  is  inu  rested  only  in  obtaining  the  predictive  Filter  coefficients  (for 
example,  to  do  time  domain  prediction  and  signal  processing),  these  results  are 
available  immediately  after  execution  of  subroutine  Pfc.  There  is  then  no  need  to 
resort  to  the  frequency  domain  routines  that  follow  Pfc;  in  this  manner,  execution 
time  and  storage  can  be  significantly  reduced.  An  additional  reduction  in  execution 
time  is  available  by  declaring  all  the  loop  counters  in  a  subroutine  to  be  INTEGER. 
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Application  to  Processes  With  Tones 

Our  first  example  is  the  two-channel  case  given  numerically  by  the  sample  values 
in  reference  7,  page  17;  reference  8,  page  K-12;  and  in  reference  9,  page  D-18.  The 
analytic  expression  for  the  autoregression  is 

x,(k)  *  .85  x,(k-l)  -  .75  x2(k-l)  +  w,(k) 

x2(k)  »  .65  x,(k-l)  +  .55  x2(k-l)  +  w2(k)  (5) 

where  {w,(k)}  and  {w2(k)}  are  uniformly  distributed,  independent  white  noise 
processes  with  zero  means  and  variances  1/12.  General  filter  and  spectral  relations 
for  moving-average  and  autoregressive  processes  are  given  in  appendix  B;  these 
general  relations  are  then  specialized  to  this  particular  numerical  example.  It  is 
shown  in  (B-31)  et  seq.  that  the  auto  spectrum  of  process  {x,(k)}  has  four  poles  and 
three  zeros  in  the  finite  z-plane,  even  though  the  two-channel  recursion,  (5),  is  only 
first-order  regressive. 

Generally,  for  a  two-channel  P-th  order  regression  and  independent  white  ex¬ 
citations  (i.e.,  Ek  *  0  for  k  >  P,  Fk  -  I  <Jk0,  and  Q(z)  ■  AI  in  (B-18)),  the  auto-  and 
cross-spectra  of  the  processes  each  possess  4P  poles  and  3P  zeros  in  the  finite  z- 
plane  (of  which  P  zeros  occur  at  the  origin).  This  is  in  contrast  fo  the  single-channel 
case,  where  2P  poles  (and  a  P-th  order  zero  only  at  the  origin)  can  occur.  This  in¬ 
creased  generality  can  be  anticipated  by  the  observation  that  whereas  a  single¬ 
channel  approximation  requires  estimation  of  only  P  parameters,  an  M-channel 
approximation  requires  estimation  of  M2P  parameters  (4P  for  the  two-channel  case 
M-2).  Of  course,  for  a  fixed  number,  N,  of  data  points  from  each  process,  the 
estimation  of  an  increased  number  of  parameters  can  only  be  done  with  increased 
variance;  this  is  a  manifestation  of  the  tradeoff  between  resolution  and  stability  that 
accompanies  all  spectral  analysis  techniques. 

The  first-order  forward  partial  correlation  coefficient  for  two-channel  process  (5) 
is 


A*0  (true)  = 


.85 

-.75 

.65 

.55 

(6) 


and  all  other  higher-order  coefficients  are  zero.  The  exact  auto  spectrum  of  the  first 
process,  {x,(k)},  is  shown  (in  dB)  in  figure  1A;  the  auto  spectrum  of  the  second 
process,  {x2(k)},  is  shown  in  figure  IB;  the  magnitude-squared  coherence  is 
displayed  in  figure  1C;  and  the  argument  of  the  complex  coherence  or  cross  spec¬ 
trum  is  depicted  in  figure  ID.  There  is  seen  to  be  a  strong  narrowband  component  at 
approximately  one-fourth  of  the  Nyquist  frequency  fN  =  (2A)*1,  where  A  is  the  time¬ 
sampling  increment  for  the  two-channel  process  (5).  This  leads  to  a  peak  magnitude- 
squared  coherence  value  of  .999013  at  2f A  =  .2459. 

The  results  of  applying  the  two-channel  spectral  analysis  program  in  appendix  A 
to  the  numerical  data  cited  above,  with  N  =  100,  are  shown  in  figure  2,  where  the 
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four  parts  of  this  figure  correspond  directly  to  those  of  figure  1 .  Pbest  turns  out  to 
be  equal  to  the  correct  value  1,  and  the  spectral  estimates  are  all  quite  good.  In  fact, 
the  estimated  magnitude-squared  coherence  reaches  a  peak  value  of  .999745  versus 
the  true  value  of  .999013. 


The  covariance  matrix  of  the  process  generated  by  (5)  is  (reference  8,  page  18, 
after  scaling  by  variance  !/*'*) 


R0(true) 


‘2.095  0.405  ‘ 

0.405  1.804. 


(7) 


The  corresponding  matrix  estimate  yielded  by  the  program  here,  based  on  the 
particular  N  ■  100  data  values  cited  above,  is 


R 


’4.62 
.  .916 


.916 
3.80  . 


forN  -  100; 


(8) 


these  values  are  approximately  2.2  times  larger  than  (7),  due  to  the  fact  that  (5)  is  a 
narrowband  process  and  the  particular  100  pairs  of  samples  used  in  the  spectral 
estimates  happen  to  lie  on  a  local  peak  of  the  instantaneous  waveforms.  Although 
the  local  estimates  of  the  absolute  power  levels  are  off  considerably,  the  estimate  of 
the  forward  partial  correlation  coefficient  is  very  good;  we  find,  instead  of  (6), 


.872 

-.770 

.634 

.560 

forN  -100. 


(9) 


Next  we  add  a  pure  tone  only  to  the  second  process  {x2(k)}  at  a  frequency  equal  to 
0.6  of  the  Nyquist  frequency,  i.e.,  at  0.6fN.  The  power  in  the  tone  is  1/512,  i.e.,  32.9 
dB  below  the  average  power,  3.80,  in  this  particular  segment  of  autoregressive 
process  {x2(k)};  see  (8).  The  resultant  spectral  estimates  are  shown  in  figure  3;  they 
are  virtually  identical  to  figure  2.  The  only  inadequacy  of  figure  3  is  that  the 
autospectral  estimate  in  figure  3B  gives  no  indication  of  the  added  tone;  of  course, 
there  should  ideally  be  no  indication  of  the  tone  in  figure  3A  for  the  auto  spectrum 
of  (x,(k)}.  The  value  of  Pbest  was  again  1,  as  determined  by  theAIC. 

When  the  tonal  power  in  the  second  process  is  increased  to  -26.9  dB,  Pbest  in¬ 
creases  to  4  (see  figure  4)  and  there  are  humps  in  both  auto-spectral  estimates  near 
the  tone  frequency  0.6fN.  The  coherence  estimates  (magnitude  and  argument)  are 
significantly  perturbed  in  a  considerable  neighborhood  of  0.6fN;  this  broad 
frequency-perturbation  width  is  due  to  a  small  value  of  Pbest  having  been  selected 
by  the  AIC. 

Increasing  the  tonal  power  to  -20.8  dB  results  in  the  estimates  depicted  in  figure 
5.  Now  there  is  a  considerable  indication  of  the  tonal  power  in  figure  5B;  however, 
there  is  also  an  undesirable  indication  in  figure  5A  at  frequency  0.6fN  in  the  auto- 
spectral  estimate  for  process  {x,(k)}.  This  “feed-across”  is  due  to  the  fact  that  we 
are  working  with  only  N  =  100  data  samples  of  each  process;  with  this  small  a  data 
set,  the  “best”  two-channel  linear-prediction  is  misled  into  an  erroneous  indication. 
It  is  important  to  observe  at  this  point  that  any  auto-spectral  estimate  based  on 
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samples  of  process  {x,(k)}  alone  would  not  give  this  tonal  indication,  since  the  tone 
is  not  present  in  this  process. 


The  coherence  estimates  in  figure  5  fare  no  better,  even  though  Pbest  «  8  now.  A 
large  magnitude-squared  coherence  value  of  0.85  is  yielded  at  frequency  0.6fN.  The 
progression  towards  poorer  behavior  is  also  present  in  figure  6,  which  employs  a 
tonal  power  of  -14.8  dB  relative  to  the  sample  power  in  {x2(k)}.  Now  the  undesired 
peak  magnitude-squared  coherence  estimate  is  0.9.  A  tonal  power  of  -8.8  dB  (figure 
7)  yields  a  near-unity  magnitude-squared  coherence  estimate  at  0.6fN,  and  a  very 
substantial  tonal  indication  in  the  auto  spectrum  of  {x,(k)},  figure  7A. 


The  situation  is  markedly  improved  if  more  data  samples  are  available.  When  N 
is  increased  to  1000,  and  data  are  generated  via  (5)  as  before,  the  sample  covariance 
for  the  particular  data  set  generated  is 


Ro 


2.60 
.  .514 


.514 
2.27  . 


for  N  *  1000, 


(10) 


for  no  tone  present.  When  a  tone  is  added  to  process  {x2(k)},  with  strength  -24.6  dB 
relative  to  the  sample  power,  2.27,  of  the  second  process,  the  resultant  spectral 
estimates  are  as  displayed  in  figure  8.  There  is  a  slight  hump  at  0.6fN  in  figure  8B, 
and  a  near-zero  coherence  estimate  at  this  frequency.  Recall  that  the  ideal 
characteristics  would  be  identical  to  figure  1  except  for  an  impulse  in  figure  8B  at 
0.6fN  and  a  very  sharp  null  in  the  magnitude-squared  coherence  at  0.6fN. 

The  results  in  figure  8  were  achieved  by  taking  Pmax  *  8,  for  which  the  AIC 
indicated  Pbest  ■  8  for  this  particular  data  set.  However,  the  AIC  is  a  very  flat 
function  of  filter  order  P  in  this  range,  and  it  is  difficult  to  justify  a  particular  value 
of  P  as  “best”.  Some  additional  information  about  the  autoregressive  portion  of 
the  observed  process,  such  as  a  limit  on  P,  could  be  useful;  for  example,  when  we 
specified  Pmax  as  1,  the  results  were  very  similar  to  figure  1.  There  was  virtually  no 
indication  of  the  tone  in  any  of  the  spectral  estimates,  even  though  it  was  in  the 
(x2(k)}  data  at  a  relative  level  of  -24.6  dB  with  respect  to  the  sample  power,  2.27,  of 
the  second  autoregressive  component.  In  fact,  the  estimated  first  partial  correlation 
coefficient  was 


A<» 


‘.8543 

.6578 


-.7394* 

.5415 


for  N 


1000, 


(ID 


which  is  very  close  to  the  true  value,  (6). 

Results  for  the  spectral  estimates  when  the  tonal  power  is  increased  to  -18.6  dB, 
-12.6  dB,  -6.6  dB,  and  -0.6  dB  are  given  in  figures  9,  10,  11,  and  12,  respectively, 
all  corresponding  to  Pmax  =  8  and  Pbest  =  8.  Even  for  the  nearly  0  dB  case  in 
figure  12,  there  is  virtually  no  indication  in  auto-spectral  estimate  12A  of  the 
strong  tonal  in  process  { x2(k)},  figure  12B.  The  magnitude-squared  coherence 
estimate  in  figure  12C  appears  to  have  developed  a  couple  of  zeros  and  poles  near 
the  frequency  f  =  0.6fN,  where  the  strong  tone  is  located;  recall  that  we  have 
4P  =  32  poles  available  in  the  approximation  for  Pbest  =  8.  Typically,  it  has  been 
observed  that  a  strong  tonal  present  in  only  one  process  manifests  itself  in  the 
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Figure  7.  Spectral 
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9A.  Auto  Spectrum  of  First  Process 


9B.  Auto  Spectrum  of  Second  Process 


9C.  Magnitude-Squared  Coherence 


9D.  Argument  of  Complex  Coherence 

Figure  9.  Spectral  Estimates  for  N  =  1000,  Pmax  =  8,  Pbest  =  8,  Tone  Power  =  -18.6  dB 


11A.  Auto  Spectrum  of  First  Process 


11D.  Argument  of  Complex  Coherence 

Figure  11.  Spectral  Estimates  for  N  *  1000,  Pmax  =  8,  Pbest  =  8,  Tone  Power  =  -6.6  dB 


12D.  Argument  of  Complex  Coherence 


Figure  12.  Spectral  Estimates  for  N  =  1000,  Pmax  =  8,  Pbest  =  8,  Tone  Power 


coherence  estimate  as  a  sharp  spike  at  the  tone  frequency.  The  change  in  argument 
in  figure  12D  in  the  neighborhood  of  this  frequency  can  serve  as  an  indicator  of  the 
number  of  poles  and  zeros  clustered  there. 

When  Pmax  was  increased  to  47,  the  AIC  yielded  Pbest  *  25  for  these  last  four 
figures.  However,  the  spectral  estimates  for  Pbest  ■  25  proved  to  be  too  spiky  and 
erratic.  Also  the  selection  of  Pbest  at  25  is  rather  tenuous,  as  figure  13  indicates;  this 
is  a  plot  of  the  AIC  versus  filter  order  P  in  the  range  (1, 47).  Although  the  absolute 
minimum  occurs  at  P  -  25,  there  are  significant  drops  in  the  curve  at  P  -  4,  6,  and 
8.  Selection  of  P  at  one  of  these  significant  drops  appears  to  be  a  promising  ap¬ 
proach,  instead  of  using  the  absolute  minimum  of  the  curve.  In  addition,  the 
flatness  of  the  curve  is  brought  out  by  observing  that  the  range  of  values  of  AIC  is 
limited  to  (-4.80,  -4.73)  for  P  in  the  wide  range  from  10  to  47.  Thus,  the  local  minor 
drops  and  nses  in  the  AIC  curves  are  not  significant;  selection  of  values  of  P 
corresponding  to  significant  decreases  seems  to  be  a  viable  approach. 


Figure  13.  Akaike  Information  Criterion  for  N  *  1000,  Tone  Power*  -12.6  dB 


The  last  example  we  consider  is  a  two-channel  process  of  N  =  64  data  points, 
composed  of  several  tones,  some  of  which  are  at  common  frequencies,  and  some  of 
which  are  not;  this  example  was  supplied  by  S.  L.  Marple  (reference  10).  In  par¬ 
ticular,  process  (x,(k)}  has  two  strong  tones  at  f  =  0.4fN  and  0.5fN,  and  a  weaker 
tone  (-20  dB)  at  f  =  0.2fN,  in  addition  to  some  low  level,  colored  background  noise. 
The  other  process  has  two  strong  tones  at  f  =  0.4fN  and  0.8fN,  and  a  weaker  tone 
(-20  dB)  at  f  *  0.2fN.  Thus  the  tonal  frequencies  common  to  both  processes  are 
0.2fN  and  0.4fN,  whereas  the  uncommon  frequencies  are  0.5 fN  and  0.8fN.  The  two 
auto-spectral  estimates  of  each  process  (obtained  via  the  single-channel,  forward- 
backward  averaging  technique  of  reference  4)  are  displayed  in  figure  14  for 
prediction  length  P  *  12  (24  poles  for  each  spectral  estimate).  There  is,  of  course, 
no  cross-feed  at  frequencies  0.5fN  and  0.8fN. 

The  spectral  estimates  of  the  same  two-channel  data  (obtained  via  the  program  in 
appendix  A  which  includes  coherence  estimation)  are  given  in  figure  15.  The  value 
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of  P  used  was  6,  which  allows  for  24  poles  in  each  spectral  estimate.  The  low 
number  of  data  points,  N  »  64,  now  allows  some  undesired  cross-feed  in  figures 
15A  and  15B  at  f  ■  0.3fN  and  0.5fN,  respectively.  This  also  shows  up  in  the 
magnitude-squared  coherence  estimate  as  two  very  sharp  spikes  at  these  two 
frequencies,  whereas  the  true  coherence  is  zero  at  these  two  frequencies.  This 
limited  capability  of  the  multi-channel  linear  predictive  technique  can  be  improved 
by  utilizing  larger  data  sets;  N  =  64  is  too  small  a  data  size  to  accomplish  a  high 
quality  result  for  a  data  set  such  as  this  with  strong  interfering  tones. 


14A.  Auto  Spectrum  of  First  Process 


Figure  14.  Auto-Spectral  Estimates  for  Multitone  Example,  N  =  64,  P  =  12 


Figure  IS.  Spectral  Estimates  for  Multitone  Example,  N  =  64,  P  =  6 


Discussion  and  Conclusions 

A  program  for  two-channel  auto-  and  cross-spectral  estimation  has  been 
presented  and  illustrated  for  cases  including  interfering  tones.  If  the  number  of 
available  points,  N,  is  too  small,  some  misleading  estimates  may  be  obtained 
because  of  cross-feed  between  the  finite  lengths  of  data  from  each  channel.  This 
cross-feed  manifests  itself  as  narrow  spurious  spikes  in  the  spectral  and  coherence 
estimates.  Choice  of  the  appropriate  value  of  filter  order,  P,  is  possible  by  ob¬ 
serving  where  the  AIC  undergoes  significant  negative  jumps,  rather  than  by  using 
the  absolute  minimum  of  the  curve.  It  can  also  be  illuminating  to  overlay  plots  made 
with  two  (or  more)  different  values  of  P,  thereby  obtaining  different  degrees  of 
resolution  and  stability  from  the  same  data  set.  The  recursive  nature  of  the  linear- 
predictive  approach  makes  this  practice  easy  to  achieve. 

A  more  fundamental  observation  about  spectral  estimation  in  general  is  now 
developed.  Suppose  we  are  given  finite  data  records  of  three  stationary  processes 
x(t),  y(t),  and  z(t),  and  we  wish  tc  estimate  all  the  auto  spectra  and  cross  spectra 
involved.  The  Blackman  and  Tukey  and  weighted-FFT  approaches  evaluate  the 
auto  spectrum  of  each  process  separately.  Thus,  the  spectrum  of  x(t)  is  estimated 
without  interference  from  y(t)  and  z(t);  the  availability  of  the  data  records  for  y(t) 
and  z(t)  plays  no  part  in  the  eventual  auto-spectral  estimate  for  x(t).  Additionally, 
the  cross-spectral  estimate  for  processes  x(t)  and  y(t)  is  independent  of  the  available 
data  on  the  z(t)  process.  Finally,  the  coherence  estimate  between  two  processes  is 
independent  of  any  additional  data  records  for  other  (statistically  related)  processes. 

On  the  other  hand,  the  generalization  (in  references  7-9)  of  Burg’s  single-channel 
linear-predictive  spectral  analysis  approach  to  the  multichannel  case  gives  auto- 
spectral  estimates  of  the  x(t)  process  that  are  dependent  on  the  available  values  of 
y(t)  and  z(t).  Also,  the  cross-spectral  estimate  between  x(t)  and  y(t)  is  dependent  on 
the  particular  z(t)  data  available.  This  procedure  can  be  poor  for  short  data  lengths 
if,  for  example,  y(t)  contains  a  strong  tone  at  f0  that  is  not  present  in  x(t)  or  z(t). 
Thus,  estimates  of  spectra  G^f),  Gxy(f),  and  Gu(f)  all  contain  tonal  indications  at 
f0  that  should  not  be  there.  These  spurious  tonal  indications  are  due  to  cross-feed 
between  the  available  finite  data  segments  of  the  various  processes. 

This  raises  the  following  questions: 

•  Should  the  estimate  of  G^f)  be  determined  only  from  the  available  x(t) 
data  record  ? 

•  Should  the  estimate  of  Gxy(f)  be  determined  only  from  the  available  x(t) 
and  y(t)  data  records  ? 

•  If  coherence  C^f^  =  0,  why  use  y(t)  to  estimate  G^f,,)  ? 

•  If  coherence  C^f^  =  1,  why  use  the  completely  statistically  dependent 
y(t)  data  to  estimate  G^Q  ? 

This  philosophy  of  discarding  "irrelevant”  data  would  be  consistent  with  the 
Blackman  and  Tukey  and  FFT  approaches.  Carrying  this  philosophy  on,  we  are  led 
to  the  following:  estimate  GM(f)  solely  from  the  x(t)  data  by  some  good  single- 
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channel  linear-predictive  technique,  such  as  forward-and-backward  averaging, 
coupled  with  an  efficient  way  of  inverting  the  relevant  matrices  (e.g.,  references  4 
and  5).  Then  estimate  cross  spectrum  Gxy(f)  or  coherence  C(f)  directly,  by  some  (yet 
unknown)  linear  predictive  technique  whose  sole  goal  is  linear  prediction  of  x(t) 
from  y(t)  and  vice  versa,  with  no  interest  in  or  diversion  from  simultaneous 
estimation  of  G^f)  or  GyJf).  By  this  means,  we  can  concentrate  on  extracting  all 
the  relevant  cross-spectral  information  with  maximum  stability  and  resolution. 
Other  cross  spectra  of  interest  between  particular  pairs  of  available  processes  can  be 
similarly  obtained,  one  at  a  time.  This  procedure  is  currently  under  investigation. 


Appendix  A 

Program  For  Two-Channel  Linear-Predictive  Spectral  Analysis 

The  program  listing  below  in  Basic  for  the  HP  9845B  Desk  Calculator  is  a 
translation  and  update  of  that  given  in  references  7-9.  A  complete  breakdown  and 
explanation  of  the  components  and  subroutines  of  the  program  are  given  in 
reference  8,  and  in  reference  9,  appendix  D. 

Inputs  required  of  the  user  are  the  integers  listed  in  lines  20,  30, 40;  they  are 

N  Number  of  data  points  in  each  process; 

Pmax  Maximum  order  of  predictive  filter  to  be  considered; 

Nfft  Size  of  FFT  to  be  used  in  spectral  analysis. 

In  addition,  the  user  must  modify  subroutine  SUB  Data(N,X(*))  in  lines  5430*3490 
at  the  end  of  the  program  to  read  in  his  own  particular  two-channel  data  sets.  Pbest 
can  be  forced  to  equal  Pmax  by  setting  Fac  *  0  in  SUB  Pcc. 

An  explanation  of  the  program  output  is  given  under  equation  (1)  of  the  main  text 
of  this  report.  A  sample  printout  for  a  short  (N  *  20)  data  sequence  that  can  be 
used  as  a  check  case  on  the  program  is  presented  following  the  listing  below.  Sample 
plots  of  the  auto-spectral  estimates  and  the  coherence  estimates  conclude  the  ap¬ 
pendix. 
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10  !  rwO-CHANNEL  LINEAR  PREDICTIVE  SPECTRAL  “NAUSIS,  TR  5501  &  5729 
20  N«20  !  NUMBER  OF  DATA  POINTS  IN  EACH  PROCESS 

30  Pm*x«6  !  MAXIMUM  ORDER  OF  PREDICTIVE  FILTER 

40  Nfft-256  !  SIZE  OP  FFT 

50  OPTION  BASE  1 

60  REDIM  Y<2,N>,Z<2,N>,Ap<Pmi'r,2,2>,Bp.P#1*  .2,2 

70  REDIM  Rr><.Pm*x ,  2, 2) ,  Ai  c  <0!  Pm*x  * ,  XI 1  <Nff  t  > ,  Y 1  lv  Nt"  t' t  • .  U  2.  ♦  o 

S0  REDIM  Y12<Nt'  ft ),  X21 '  Nf  f  t  )  .  Y21  <!Nf  f  t  >,X22<Nf  ft  ' ,  Y22-  NU  *  .■ 

90  DIM  Y<2,l00e>,Z<2,108O>,Ap<.25.2,2>,Bp'>25,2.2' 

100  DIM  Rn<25,2,2),Atc<0;25),Xll<1024>,Yll( 1024). X12<  1024  • 

110  DIM  Y12<1024),X21<1024',Y21<1024),X22<1024>. Y22< 1024) 

120  DIM  Av*>2),Ub«*t  <2, 2) ,  U<2, 2) ,  V<2,  2  > ,  U  i  •  2, 2) ,  Vi '  2 , 2> ,  A.'  2, 2, 

130  DIM  B<2,2>,R<2,2>,U*<2,2>,Wb<2.2>,Wc<2.2),Md<2.2>.W«<2,2> 

140  PRINT  "NUMBER  OF  DATA  POINTS  IN  EACH  PROCESS  N  «";N 
150  PRINT  "MAXIMUM  ORDER  OF  PREDICTIVE  FILTER  Pm**  -“;Pm*x 
160  PRINT  "SIZE  OF  FFT  Nfft 
170  PRINT 

180  CALL  D*t*<N,Y<*>) 

ISO  PRINT  "PROCESS  NUMBER  1" 

200  FOR  I«1  TO  N 
210  PRINT  Y ( 1 | I > ; 

220  NEXT  I 

230  PRINT  LINO) 

240  PRINT  "PROCESS  NUMBER  2" 

250  FOR  l-l  TO  N 
260  PRINT  Y<2,I>; 

270  NEXT  I 

260  PRINT  lIN<2) 

290  CALL  Pc  c  <■  N ,  Pm  *x ,  A»<* »  *  > ,  W*«.  ,  nt  *  • ,  Hi  *  ■ ,  WO 1  »  1 ,  W* «  *  > ,  R  ■  *  ■ ,  U  ■  *  * ,  V 

<  *) ,  Ai  c  <  *> ,  Pb**t ,  Ub«*t  <  *) ,  Ui  <*) .  Vi  • * ' ,  A'  *  > ,  8 ■ e  > , Ap ■ ♦  .  Ep-  •* 1  1 
300  PRINT  "MEANS  OF  INPUT  DATA  <A-,«>:'‘ 

310  PRINT  Av«<l) 

320  PRINT  Av«<2)  * 

330  PRINT  LINU) 

340  PRINT  "COVARIANCE  MATRIX  OF  INPUT  DATA  <R>:".R'*' 

350  PRINT  " AKAIKE  INFORMATION  CRITERION!" 

360  PRINT  *  P  Ai c  <P> " 

370  IMAGE  3D, 4<4X, M.  9DE) 

300  FOR  P"0  TO  Pntx 

390  PRINT  USING  370;P,Aic<P> 

400  NEXT  P 

410  PRINT  LINU) 

420  PRINT  “Pb*st  «";Pb«*t 
430  PRINT  LINU) 

440  PRINT  "Ub**t! ",Ub«*t<*> 

450  PRINT  "FORWARD  PARTIAL  CORRELATION  COEFFICIENTS!" 

460  PRINT  "  P  Apvl.l)  Ap<2,  1  *  ApU,2> 

Ap< 2 , 2)  “ 

470  FOR  P»1  TO  Pm»x 

430  PRINT  USING  370JP, Ap<P, 1, 1 ), Ap> P,  2,  1 >.  Ap- P.  1 , 2 ■ . Ap  P.2.2< 

490  NEXT  P 

500  PRINT  LINU) 


Reproduced  from 
-°esl  available  copy 


A-2 


IR  6533 


310  PRINT  “  BACKWARD  PARTIAL  CORRELATION  COEFFICIENT'S:" 

320  PRINT  “  P  BpO,l>  Bp1 2, 1 )  BpO,2) 

Bp<2,2>“ 

330  FOP  P*1  TO  Pioax 

340  PRINT  USING  370; P , Bp<P, 1 , 1 ) , BpCP ,  2,  1 > ,  Bp- P , 1 . 2 • , Bp ' P , 2 , 2) 

330  NEXT  P 
560  PRINT  LINO) 

370  IF  Pb*4t«0  THEN  890 

3S0  CALL  PfctP»*xfPb**t,R<*>1Ap<*),Bp<:*'>,  »*<*'.  Mb'.  ♦>*Wc<*>.Wd<*>,Rn<*>,Rllf  R22 
,  R12> 

390  PRINT  "FORWARD  PREDICTIVE  FILTER  COEFFICIENTS  FOR  Pb*»t ! ” 

600  PRINT  *  P  ApO,l>  Ap'2,l>  Ap<l,2) 

Ap(2, 2 ) " 

610  FOR  P«1  TO  Pb*it 

620  PRINT  USING  370;  P ,  Ap<P,  1 , 1 )  ,  Ap<P,  2.  t  )  ,  Ap(.P,  1 , 2)  ,  Ae  kP  ,  2, 2) 

630  NEXT  P 

640  PRINT  LINO) 

630  PRINT  "NORMALIZED  CORRELATION  MATRICES  (Rr.):1* 

660  PRINT  "DELAY  AUTOll  CR0SS2I  CR0S312 

AUT022" 

670  PRINT  USING  370; 0, R< 1 ,  1  > , R(2, 1 ) , P< 1 , 2 ) . R<2, 2) 

680  FOR  P«l  TO  Pmax 

690  PRINT  USING  370; P, Rn<P, 1 , 1 > , Rn<P, 2,  l >  ,  Rn< P, 1 , 2 > , Rrv P, 2, 2) 

700  NEXT  P 
710  PRINT  LINO) 

720  CALL  P«f  t  f<Pb«*t,  Nff  t,Ap<*>,  XI  1<*),Y1H*),X12<*  -.Y12'  *  > ,  X21  <  O ,  Y21  <  *  ) ,  X22' 
*>,Y22<*>> 

730  CALL  Sd(*<  Nf  f  «• ,  Ub*S t  <  *  )  ,  U*(  *  )  ,  Ub <  * )  ,  Wc  •  *  >  ,  Ud<  *  > .  W«  •  *  >  ,  X 1 1  (  *  > ,  ( 1 1  >.  * ) ,  X  1 2  <  *  > , 
Y 1 2 <  *  > ,  X2 1  v  *  >  ,  Y2 1  ,  X22  <  *  > ,  Y22 <  ♦  ; ,  $  1 1 ,  S22 ,  S 1 2  ) 


740 

M 

PRINT  "SPECTRAL  DENSITY  MATRIX  AND  COHERENCE, 

FROM  ZERO  FREQUENCY  -BIN  1>: 

730 

PRINT  “  BIN  AUTOL1  AUT022  RE<CR0SS1 

ARGUMENT" 

2)  IM«.  CP0SS12)  MAG  SO  COH 

760 

IMAGE  3D,3(M.6DE, 1X),M.6DE 

770 

FOR  1-1  TO  30 

780 

L«I 

790 

IF  I < 1 6  THEN  840 

800 

IF  I>16  THEN  830 

810 

PRINT  "**#" 

820 

GOTO  830 

830 

L«I*Nfft/2“29 

840 

PRINT  USING  760;L(XlltL), X22<L ) ,X12'L • ,Y1 2  >  L) , 

VI 1 <L  >. Y22(L> 

830 

NEXT  I 

360 

PRINT  L I N <  1  ) 

870 

PRINT  "TRAPEZOIDAL  SUMS  OF  SPECTRA: " 

880 

PRINT  SI  1 , S22, S12 

890 

PRINT 

900 

PRINT  "COVARIANCES  OF  INPUT  DATA: " 

910 

PRINT  R1 1 , R22, R12 

920 

PRINT  LINO) 

930 

CALL  AcdKNfft  ,XU<-*',X12<*),Y12<*>,X21<*>,  Y21< 

*>,'.<22’  -  ■  ,  X  1  1  m  1  ,  X  2  2  m  1  ,  '1  1  rri  0 , 

X22n0  > 

940 

N1 «Nf  f t ♦ 1 

950 

N2»Nfft^2 

960 

N22«N2*2 

TR  6533 


970  PRINT  "ALIASED  NORMALISED  CORRELATION  MATRICES!" 

980  PRINT  “DELAY  AUTOll  CR0SS21  CROSS12 

AUT022“ 

990  PRINT  USING  370;  0,  X 1 1  <N22>  ,  X2 1  <  1  >  ,  X2 1  >.t  '> ,  X22' .X22  ■ 

1000  FOR  I-l  TO  27 
1010  L-I 

1020  IF  I<  16  THEN  1070 
1030  IF  I  >16  THEN  1060 
1040  PRINT  “#**“ 

1030  GOTO  1080 
1060  L-I*N2-29 

1070  PRINT  USING  370;  L,  X 1 1  < N22*L> ,  X2 1  <N1  -L  ■ ,  X2 1  ->  1+L  >  .  X22«.N22*L  ) 

1080  NEXT  I 

1090  PRINT  USING  370;N2-l,XUiiil,X21(N22),X21<.N2;,X22ro' 

1100  PRINT  USING  370;N2,Xll»0,X21<N2>l>,:<21<Ni*l  • ,  X22r»0 
1110  PRINT  LIN<2> 

1120  PRINTER  IS  0 

1130  PRINT  “AUTO  SPECTRAL  DENSITIES  IN  DB :  “ 

1140  PLOTTER  IS  “GRAPHICS* 

1130  GRAPHICS 

1160  SCALE  0,N2,-3,0 

1170  GRID  N2'4,  1 

1180  PENUP 

1190  FOR  1-0  TO  N2 

1200  PLOT  I ,  LGT  <  XI  1<I*D) 

1210  NEXT  I 

1220  PENUP 

1230  FOR  1-0  TO  N2 

1240  PLOT  I ,  LGT  C  X22( I  + 1 )  ) 

1230  NEXT  I 
1260  PENUP 
1270  DUMP  GRAPHICS 
1280  PRINT  LINO) 

1290  PRINT  “MAGNITUDE  SQUARED  COHERENCE  AnD  ARGUMENT" 

1300  PLOTTER  IS  “GRAPHICS* 

1310  SCALE  0,N2,0,1 
1320  GRID  N2'4,.23 
1330  PENUP 
1340  FOR  1-0  TO  N2 
1330  PLOT  I,Y11CIM> 

1360  NEXT  I 
1370  PENUP 

1380  SCALE  0, N2, -PI i PI 
1390  FOR  1-0  TO  N2 
1400  PLOT  I ,  Y22< 1  +  1 ) 

1410  NEXT  I 
1420  PENUP 
1430  DUMP  GRAPHICS 
1440  PRINT  L I N  (  4  ) 

1450  PRINTER  IS  16 
1460  END 
1470  ! 
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1480  SUB  Pce<N,P»4x,Y<»>,Z<»>,Av«'*>.W4<*.>,WbO;, ,  Wd'  ■*  ;• ,  w*<  *> ,  R-;  *  > ,  U 
*),flicf*)iPb«»t,Ub*»t<*),Ul  <*>,Vl<#>,A<*>,B<*>,Apc.»»,Bp<*>i 
14-90  U«INT<  1 . 5*SQR<N>  > 

1500  IF  P»4x,»l4  THEN  1520 

1510  PRINT  "Pi»4x  ■*;P«.4x;“IS  TOO  LARGE  FOR  N  -“;N;"  SEARCH  LIMITED  TO  P  » 
1520  1 4"M I N(  I  4, Pm4x ) 

1530  F4C-S/N  I  F 4C-0  WOULD  FORCE  Pb«tt  EQUAL  TO  Pi»4x 
1540  MAT  Au«“RSUM‘.Y> 

1550  MAT  Au*«Au«x(N> 

1560  Al«Av#U> 

1570  A2«Av*<2> 

1580  FOR  1-1  TO  N 
1590  Y<1, I >»y < 1 , I)-A1 

1600  Y<2, I)«Y<2, I>-A2 

1610  NEXT  I 
1620  MAT  Z«Y 

1630  CALL  Auto<2,N-l,Y<#),Wc<*)) 

1640  CALL  Auto< 1, 1 , Y< *> , Md< * > ) 

1650  CALL  Auto<N,N, Y<*),W*<*> ) 

1660  MAT  H4-Uc+H« 

1670  MAT  Wb«Uc*Wd 
1680  MAT  R>Ub+U« 

1690  MAT  R«Rxf N) 

1700  MAT  U"R 
1710  MAT  V-R 

1720  CALL  Cro»*<2, N, YC *) ,Z(#),Mc(*)) 

1730  A«c<0>-LOC(DET<U>) 

1740  Aicain«Aic <0> 

1750  Pb«sta0 
1760  MAT  Ub*»t-U 
1770  FOR  P"1  TO  1 4 
1780  MAT  VI «INV< V) 

1790  MAT  Wd«V1*Wb 
1800  MAT  Wb-Ud 
1810  MAT  Ui-INV<U) 

1820  MAT  Ud-U4 
1830  MAT  W4«Ud*Ui 
1840  MAT  Uc«Hc*<2) 

1830  CALL  Solu*<W4<*>>Ub<*),Wc<*),Wd<»>,W*^*'  • 

1860  MAT  A-Uc»V< 

1870  MAT  Ud«TRN<Wc> 

1880  MAT  B-Hd*Ui 
1890  Ap<P, 1 , 1 )«A<  1(1) 

1900  Ap<P, l,2)«A<l,2) 

1910  Ap<P,2,  1>-A<2, 1) 

1920  Ap<P,2,2>-A<2,2> 

1930  Bp(P,  1,  1)-1U,  1) 

1940  lp<P,l,2>-*<l,2> 

1950  Bp<P,2,  1)»*<2, 1) 

1960  Bp<P,2,2>»B<2,2) 

1970  MAT  U«>A«Ud 
1980  MAT  U>U-U« 

1990  MAT  U«-B*Uc 
2000  MAT  V»V-U* 


2010  Pic<P>-LOGtDET<U>)*F*c*P 
2020  IP  Pi  c  -  P  j  >»Picir.i  n  THEM  2060 
2030  Pi  cm  n-P  i  c  <P> 

2040  Pb«*t-P 
2030  MPT  Ub#st -U 
2060  IP  P-I*  THEN  2220 
2070  L-PM 

2000  POP  K-N  TO  L  STEP  -1 
2090  Pl-YCl.tO 
2100  P2-Y<2, K) 

2110  81-Z(1,K-1> 

2120  B2-Z<2,K-1> 

2130  Zd,tO-Bl-B<l, 1>*P1-B< 1,2) *02 
2140  Z<2,K)-B2-8(2, 1>*H1-B<2, 2 >*02 

2130  Y<1,K>»01-0<1, 1)*B1-P<1,2^*B2 

2160  Y<2,K)-P2-P<2, 1)*B1-P<2,2)*B2 

2170  NEXT  K 

2130  CPLL  0uto<P*2,N,Y<*>,  «*<*>> 

2190  CPLL  Puto<P*l ,N-1 ,  Z<*  > , Mb<*>  > 

2200  CPLL  Cros»<P*2,N,Y<*>,Z<*>,He<**> 

2210  NEXT  P 

2220  01».5*<Ub«*t< .l,2)*Ub«*t<2, 1>> 

2230  Ub**t<l,2>-Ub**t<2, 1>-01 

2240  SUBEND 
2230  ! 

2260  SUB  Pf c  <Pn>*.'s,Pb**t ,  R<*> , Pp» *> , Bp<  *j  , Hk  -  ,kb'»  .  Uc  ■' *  • ,  Wdd  > ,  Rn< * > ,  R 1 1 ,  R22 , 

R12> 

2270  Rrul,  1 ,  1  >-0pd, 1,  !>•»< 1, 1 **Pp' l, 1,2  -*P  -  2.  1  • 

2280  Rn<l,  l,2^«Pp<l, 1, 1 >•*< 1 ,2>*Pp  1, 1,2  -P'  2,2  * 

2290  Rru'  1 , 2,  1  >-ftp< 1,2, l>*»<  1,  1 >*Pp> 1,2,2>*P- 2.  I 
2300  Rn<  1 , 2, 2>-Hp< 1 , 2, 1 >•»< 1 , 2  >*Ppt 1 ,2,2/*R'  2,2 
2310  POR  P-2  TO  Pb«st 

2320  Med,  l)-Pp<P,  1,  t  >•»<  1 ,  1  >*0p*  P,  1 ,2>*»<2.  1 
2330  Me <  1 , 2 > -Pp«.P,  1 ,  l  >*P<  1 , 2  > *Ppf P,  1 , 2>*R<  2, 2 > 

2340  Me <2,  1  >“0p<P, 2, 1  >•*<  1 , 1 >*Pp^P, 2, 2^*R  2.1) 

2330  Wc<2,2)-Hp<P,2, 1>*R< 1 , 2> *Pp< P, 2, 2 >•»< 2 . 2 > 

2360  FOR  L-l  TO  P-1 
2370  Ib-P-L 

2380  U«d,  l>-0p<P,  1,  U*Bp<  lb,  1 ,  1  >*Pp'P,  1 ,2  *Bp-  Ib,2, 1 > 

2390  U«<t,2>-Pp<P, l, l>*BpiIb, 1 . 2)*0p<P, 1 , 2 ■ -Bp' lb, 2, 2  • 

2400  U*<2,  1 )-Pp<P,2, P*Bp(  lb, I,  l  )*Pp<  P,2,2>*Bp<Ic,2, 1 

2410  M4<2,2>-Pp<P,2,  l>*BpUb,  1 , 2>*0p<P,2,2  •*Bp\Ib.2,2 

2420  U«d,  l)*Pp<L, 1, 1>-U««1,  1> 

2430  M*d,2>-Pp<L, 1,2>-M*C1,2> 

2440  M*<2,  l)-Pp<L,2, 1>-M*<2, 1> 

2430  U*<2, 2>-Pp<L,2,2>~M*(2,2) 


2460  Wb<  1 , 1  )«Bp<P,  1 ,  1  >*flp<l.,  1 ,  1  .J  +  Bp'.P,  1 , 2  '*fip<L.  2,  t  1 
2470  Wb<l, 2>-Bp<P, 1, l)*ftp<L, l,2)*8p<P, 1 ,2>*ftp<'_, 2,2  - 
2480  Hb<2,l)-Bp<P,2, l>*flp(L, 1, 1 ^Bp'P, 2, 2) *Hp< L, 2 , 1 > 

2490  Wb<2,2)«Bp<P,2,  l)*Ap<l,  l ,  2)+Bp<P,  2, 2)  **p'.L.  2. 2> 

2300  Bp<Ib, 1, l)»Bp<Ib, l, 1>-Wb<l, 1) 

2310  Bp< lb, l,2)»Bp< lb, 1 , 2)-Wb< 1,2) 

2320  8pUb,2,  l>«lp<Ib,2, 1)-Wb<2, 1) 

2330  Bp< lb, 2, 2)»Bp< lb, 2, 2)-Wb<2 , 2) 

2340  fip<L, 1 , 1 >*W*( 1,1) 

2530  flp<L, 1,2>-U*<1,2) 

2360  Rp<L,2, 1 )*U*(2, 1 > 

2370  Rp<L,2,2)»U*<2,2) 

2380  Ud<l,l)*U«<l,  l)*Rn<Ib,l,l)*W*<l,2)*Rru  Ib,2.  1> 

2390  Hd<  1,2>«H*<1,  1  )#RnMb,  1,2) ♦U*<1, 2)*Rr><  Ib,2,2'> 

2600  Wd<2, 1 )"U&(2, 1 )*Rn< lb, 1, 1 >*U*<2, 2>*Rn< Ib,2, 1 » 

2610  Md<2,2)»M*<2,  1  >*Rn<  lb,  1 , 2  > ♦  «*<  2 , 2 ) «Rn<  Ib,2,2; 

2620  MBT  Wc  «Mc  +Wd 

2630  NEXT  L 

2640  Rn<P, 1 , 1 )»Wc  < 1 , 1  ) 

2630  RruP,  1,2)-Uc<  1,2) 

2660  Rn<P,2, l)>Mc<2, 1) 

2670  Rn<P,2,2)-Hc<2,2) 

2680  NEXT  P 

2690  POR  P-Pbcst>l  TO  Pfc** 

2700  HOT  U«>ZER 
2710  FOR  L-l  TO  Pb#*t 
2720  Ib-P-L 

2730  Wb< 1, l)-Ap<L,  1 , 1 )#Rn( lb, 1 , 1 ) ♦ftpCL, 1 , 2  ■  *Rn< Ib,2, l ) 

2740  Ub<  1 , 2)*flp(L,  1 ,  1  )«Rn(  lb,  1 , 2'>«-flp<L,  1 ,  T^Rn'  lb,  2. 2  ' 

2730  Mb<2, l)»flp<L,2, l)*Rn< lb, 1 , 1 )*flp<L, 2, 2) *Rn< lb. 2. 1 > 

2760  Wb<2,2)«flp(L,2,  l)*Rn<Ib,  1 , 2) ♦PpCL,  2, 2  > -*Rn'  Ib.2.2; 

2770  MAT  U*-U**Ub 

2780  NEXT  L 

2790  Rn<P,l,l)-H*<l,  1) 

2800  Rn<P, 1,2)-H*<1,2) 

2810  Rn<P,2,l)-M*<2,  1) 

2820  Rn(P,2,2)-U*<2,2) 

2830  NEXT  P 
2840  Rll-R(l,l) 

2830  R22*R<2,2> 

2868  R12»R( 1 ,2) 

2870  T«SQR<Rl 1*R22) 

2880  R<1,  1)»R<2,2)"1 

2890  R<1,2)"R<2,1)«R12//T 

2900  FOR  P«1  TO  P»*x 

2910  Rn(P, 1, t)-Rn<P, l, 1)/Rll 

2920  Rn(P, l,2)«Rn<P, 1,2)/T 

2930  Rn<P,2, l)-Rn<P,2, l)/T 

2940  Rn(P,  2, 2)"Rn<P,  2, 2)sR22 

2930  NEXT  P 

2960  SUBEND 

2970  ! 


2980  SUB  P«ftf<Pb*it,Nfft,Ap<*>tXll<*>,Yll<*'.  '  12<.*  » , l2<  *  ,  X21 C  *  >  ,  Y2l  <*  > ,  X22<  * 
), Y22t*>> 

2990  X11U)-X22<1>*1 

3000  FOR  L«1  TO  Pb#st 
3010  XI 1  <L  +  1 )*-Ap<L,  1,1) 

3020  X12<L*1>— ApiL,  1,2> 

3030  X21 (L+l )"-Ap<L, 2,1) 

3040  X22<LM>  — Ap<L,2,2) 

3030  NEXT  L 

3060  CALL  Fftl0<Nfft,Xll<*),Yll<jO> 

3070  CAUL  Fftl0<Nfft,X12<*),Y12*.*)  > 

3A80  CALL  Fftl0<Nfft,X21<.*),Y2l<.*>> 

3090  CALL  Fftl0<Nfft,X22<*),Y22<*)) 

3100  SUBEND 
3110  ! 

3120  SUB  Sd«»<Nf  f  t ,  Ub#»t  <  *)  ,  M*<*> , Mb<*) , Me  <  *> , Wd«  #  ■ ,  W«  •;*)  ,  XI 1  *  *  )  ,  Y1 1  <  *  > ,  X12<  * ) ,  Y 
12<  *) , X21 <  *> , Y21 <*> ,  X22<  *  > , Y22<*> , SI  1 , S22, SI  2 ■ 

3130  T«2-Nfft 
3140  S11-S22-S12-0 

3130  J«Nfft/2*l 
3160  FOR  L-l  TO  J 
3170  W*<1, 1>-X22<L> 

3180  U*<1,2)—  X12<L) 

3190  U*<2, 1 >»-X2l <L) 

3200  Ua<2,2)-X11<L) 

3210  Wb< 1 , 1 )»Y22(L) 

3220  Wb<  1 ,2)"“Y12CL) 

3230  Wb<2, 1 )«-Y21  <L) 

3240  Ub<2,2)«Yll<L> 

3230  T**DET (U*)-DET<Ub) 

3260  Tb*W*<l,l  )*Ub<  2 ,2)+W»<2,2)*Wb<  1,1)-M*-  l,2/-*Wb-2,  1  <-Wv  2,  !>*WbU,2> 

3270  T*«T/<T**T**Tb*Tb> 

3280  NAT  Mc-TRN(Ua) 

3290  NAT  Ud-Ub«st*Uc 
3300  NAT  Uc-Ub*Ud 
3310  Tb-Hc<l,2)-Mc<2, 1) 

3320  NAT  Mc-U«*Ud 
3330  NAT  Ud-TRN(Ub) 

3340  NAT  U*«Ubcst*Ud 
3330  NAT  Ud>Ub*U« 

3360  NAT  Uc-Hc+Ud 

3370  Yll<L)»<Uc<l,2)^2>Tb*Tb)^<Uc<i. l)*Uc* 2.2<> 

3380  Y22<L)"FNArg<Uc  < 1 , 2) , Tb) 

3390  XI 1 <L)"T4*Uc  < 1 , I ) 

3409  X22<L)"T**Uc (2,2) 

3410  X12<DaT 4*Uc  <  1 , 2) 

3420  Y12<L)«T**Tb 

3430  SI 1*S1 1*X11<L) 

3440  S22-$22*X22<L> 

3430  S12-S12«X12CL) 

3460  NEXT  L 

3470  Sll-Sll-.5#<X11(1)+Xtlf J)) 

3480  S22«S22-.5*<X22< 1 >*X22< J) ) 

3490  S12«S12-.3*<X12U  >*X12<J>  ) 

3300  SUBEND 
3310  I 


3520  SUB  Ae»<Nfft,XlH*>,X12<*>f  Y12',  *>  ,  X21  <  *>  ,  r-21  22<.  *  • ,  XI 1  Ml ,  X22mt ,  XI  1*0,  X 

22m0) 

3530  N2-Nfft/2 
3540  N21-N2+1 

3550  N22-N2+2 

3560  FOR  L* I  TO  N2 
3570  X21 <L)“. 5*X1 l CL) 

3500  Y21 (L>*. 5*X22<L) 

3590  X21 <N2*L)*. 5*X1 1 <N22-L> 

3600  Y21<N2+L>-.5*X22<N22-L> 

3610  NEXT  L 

3620  CALL  Fftl0<Nm,X21<*>,Y21<*>> 

3630  T«>l/X21<0 

3640  Tb-1/Y21(i> 

3650  T-SQR<T4*Tb> 

3660  XU<N22>-X22<N22>-1 

3670  FOR  L-2  TO  N2-1 
3600  XI 1 (N21+D-X21 <L)*T* 

3690  X22<N21*L>-Y21<L>«Tb 

3700  NEXT  L 

3710  XI lal-X21 (N2)*T* 

3720  X22m1-Y21 (N2)*Tb 

3730  Xll40-X21<N21)*T4 

3740  X22m0-Y21 (N21 )*Tb 

3750  X21<1)-.5*X12<1>*T 

3760  Y21<1>  — .5#Y12<1)#T 

3770  FOR  L-2  TO  N2 
3700  X21<L>-Xt2a>*T 

3790  Y21<L>—  Y12<L)*T 

3000  X21<N2+L)-Y21(N2>L)-0 

3010  NEXT  L 

3020  X21<N21>-.5*X12<N21>*T 

3030b  Y2KN21)  — . 5»Y12<N21 >*T 

3040  CALL  FfilO<Nfft,X21<*),Y2l<-)) 

3050  SUBENO 
3860  I 


3870  SUB  Cro**<Nl,N2,A<*>,B<*>,C' ♦>> 

3880  S11-$12»S21»$22«0 

3890  FOR  K-Nl  TO  N2 
3900 

3910  A2"fl<2,  tO 

3920  BlaB< 1 1 K  —  1 > 

3938  B2*B<2,K-n 

3940  Sll«$ii*Al*Bl 
3930  S12-S12*A1*B2 

3980  S21*S21*A2*B1 

3970  S22-S22+A2*B2 

3980  NEXT  K 
3990  C<l,l)-Sli 
4800  C< 1 ( 2>"S12 
4010  C(2,1)>S21 

4020  C<2,2>-S22 

4030  SUBEND 
4040  ! 

4030  SUB  Rut o<Nl , N2,fl<*>,B<*>> 

4080  SU-S12-S22-0 

4070  FOR  K*N1  TO  N2 
4080  Al«fl<lfK> 

4090  R2«R<2,K> 

4100  SU-S11+R1*R1 

4110  Sl2-S12fRl*R2 
4120  S22-S22*R2*A2 

4130  NEXT  K 
4140  B  < 1  *  1 >*S1 1 
4130  B<l,2)-B<2, l)«S12 

4180  B<2,2>«$22 

4170  SUBENO 
4130  I 

4190  SUB  Solv«CI4«(*>,Ub<*>  ,  He  <  *  J  , Wd< ♦ >  ,  14*  (  *  >  * 
4200  T*«W*<1, l>*U*<2,2)*Wb<l, t>-»Ub<2,2> 

4210  Tb«DET<W*)-DET<Mb> 

4220  NAT  Wd«Wc*Wb 
4230  U*<1, 1)-M*<2,2) 

4240  U«<  1,2>— W*<1,2> 

4230  Ut<2( 1 >«-H*v2f  1  > 

4280  U*  <  2) 2>"W*( 1,1) 

4270  MAT  U**W**Mc 
4280  NAT  Ud-Wd+U* 

4290  NAT  Wb-Wb*<T*> 

4300  Mb< 1 , 1 )«Wb< 1 , 1 )*Tb 
4310  Ub(2,2>-Ub<2,2J«’Tb 

4320  NAT  U#-INV<Mb> 

4330  NAT  Wc«Wd*W* 

4340  SUBENO 
4330  * 


4360  SUB  Fftl0<N,XM>,Y<*>>  !  N  <■  2^10  ■  1024,  N»2  INTEGER 

4370  DIM  C<1!237> 

4380  DATA  1, .999981175233, . 999924701839, . 99983058 1 796. .  9996983 13696, . 9993294 1 73 
01 , . 999322384388, . 999077727733, . 998793436203, . 998473330573,  .  998 1181 12900 
4390  DATA  . 997723066644,  .  997290456679,  .  99632029929 l , . 9963 126 1 2 183, . 9937674 1 4468 
, .993184726672, .994364370734,  .993906970002, .993211949233.  .992479334399 
4400  BATA  .991709733669,  .990902633428,  .990053210262, .939176309963, .983237367731 
, .987301418138, .986308097243, .983277642389, .984210892  387,  .983103487431 
4410  DATA  .981963869110, .980783280403, .979369^63683, . 9733 1 7370720, . 977028 1 42638 
, .975702130039, .974339382786, .972939932206, .971303890986,  .970031233193 
4420  DATA  .968322094274, .966976471043, .963394441693, .963776063793, .962121404269 
, .960430519416, .958703474896, .936948333732, .935141163386, .933306040334 
4430  DATA  .951435020969,  .949323180393, .94  7585591013, . 943607323381 , . 943393438 1 62 
, .941344063183, ,939439223602, .93733901 1913, . 933183509939,  . 932992798833 
4440  DATA  .930766961079, .928306030473, .926210242138, .923379332311, .921314039342 
, .919113831690, .916679039921, .914209733704, . 911 706032005,  . 909167963091 
4430  DATA  .906393704313, .903989293123, .901348347046. . 898674463694, . 893966249736 
, .893224301  196, .890448723243, .837639620403, . 884797098431,  .331921264348 
4460  DATA  .879012226429,  .876070094193, .873094973418, .370036991  109, .867046243316 
, .863972836122, .860866938638,  .837728618000, . 354337988363,  .351333193103 
4470  DATA  . 848 1 20344803,  .  844833363230, . 94 1 334977437 , . 838224703333, . 834362874986 
, .831469612303, .828043045238,  .824339302783, . 32 1 18251 4991 ,.  8 1 7584813132 
4480  DATA  .014036329706, .810437198233, .306347533344, . 80320733 1 48 1 , . 799337269 1 08 
, . 793336904609, . 792106377300, . 798346427627, . 734356597136,  .  730737223372 
4490  DATA  . 776888463673,  .  773010453363, . 769103337646, . 763167263622, . 761202383434 
, .737208846306, .733186799044, .749136394323, . 745037785441 ,  .  740931125333 
4-0 1  DATA  . 736816563877,  .  732634271672', .  728464398448 . . 724247082931 , . 720002307961 
, . 715730823284, .711432193743, .707106781 187, . 702754744437,  . 698376249409 
310  DATA  .693971460890, .609340344737, .683033667773, . 638600997793, . 676092703373 
, .671338954847, .666999922304, .662413777590, . 637806693297, . 633172842954 
4520  DATA  .648514401022, . 64383 1342890 ,. 639 1 24444864 , . 634393284 1 64, . 6296382389 1 3 
, .624359488142,. 62003721 1763,  .613231590581, .610382806276, .60531  1041404 
4330  DATA  . 600616479384,  .  393699304492, . 39073970 1 339 , . 533797837436, . 3808 1 3938096 
, .373308191418, .370780743887,  .363731810784, . 360661376197,  .  333370233020 
4540  DATA  . 330437972937,  .  343324988422,  . 340 1 71 472730. . 53499761 9887, . 329803624686 
, .324389682678, .319333990166, .314102744193, .30383014234  3,  .  303338383726 
4330  DATA  .498227666973, .492898192230, .487550160143, .432133772079, .476799230063 
, . 471 396736826, .463976493768, .460338710958, . 453033387126,  . 449611329633 
4560  DATA  . 444122144370, . 4336 1 6233539, . 4330933 1 3353 , . 427333093430, . 422000270800 
, .416429360098, .410843171038, .403241314005, . 399624199846,  . 393992040061 
4370  DATA  . 388343046699, . 382683432363 , . 3770074 l 02 1 6 , . 371 3 1 7193932, . 363612997305 
, .339393036333, .334163323420, .348418680249, . 34266071 7312,  .336889833392 
4380  DATA  .  331106303768,  .323310292162, .  319502030316, . 31368 1 740399, . 307849640042 
, :302003949319, .296130888244,  .290284677234, . 2844073372 1 1 ,  273319689383 
4390  DATA  .272621333430,  .266712737473, .2607941  17915, .254663659603, .248927603746 
, .242980179903, .237023603994,  .231038108231, .  22308  391  1 360,  .219101240157 
4600  DATA  .213110319916,.  20711  1376192, . 201  184634942, . 1 930903220 1 6 , . 1 89063664 130 
, . 183039887935, . 177004220412, . 170961388760, . 164913120490, . 138838143334 
4610  DATA  .  132797133238,  .  146730474433, .  140653239333, . 1 34530708307 ,. 1 28499 1  1 0794 
, . 122410673199, .  116316630912, .  1  10222207294, .  1 04 1 2 1 63 3872, . 980 1 7 1 403296E- 1 
4620  DATA  . 9 1908936497 IE- 1 ,  .  857973 1234  44E-1 , . 7968243797 14E- 1 ,  .  733643633997E- 1 , . 
6744391 93637E- 1 , . 613207363O22E- 1 , . 33 1952443497E- 1 . . 490676743274E- 1 
4630  DATA  .  429382369349E- 1 ,  .  3680722294 1 4E- l , . 3067430 3 1 766E- 1 , . 2434122S3229E-1 , . 
184067299038E-1, . 122713382837E- 1 , . 6 1 35384649 1 3E-2 , 0 
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4640  READ  C<*> 

4630  M-1024/N 

4660  FOR  1-0  TO  N/4 
4670  c<i*i>-ctn#i*i> 

4600  NEXT  I 
4690  Nl-N/4 
4700  N2-Nl*l 
4710  N3-N2* 1 

4720  N4-N 1 *N3 

4730  Log2n-INT<1.4427*L0C<N>*.3> 

4740  FOR  1 1-1  TO  Log2n 
4730  J2-2-<Lofl2n-Il > 

4760  13-2*12 

4770  I4-N/I3 

4780  FOR  13-1  TO  12 
4790  I6-<I3-1>*I4*1 

4000  IF  I6< -N2  THEN  4840 
4810  N6-“C<N4-I6) 

4820  N7--C<I6-N1> 

4830  GOTO  4860 
4840  N6-C< 16) 

4e30  N7— C<N3-I6> 

4860  FOR  17-0  TO  N-I3  STEP  13 
4870  18-17*13 

4880  19-18*12 

4890  N8-X< I8)-X< 19) 

4900  N9-Y< I8)-Y< 19) 

4910  X< I8)-X< I8>*X< 19) 

4920  Y<I8)-Y<I8>*Y<I9> 

4930  X< I9)-N6*N8-N7*N9 
4940  Y< I9)-N6*N9*N7*N8 
4930  NEXT  17 
4960  NEXT  IS 
4970  NEXT  II 
4980  I l-Log2n*l 
4990  FOR  12-1  TO  10 
3000  C(I2)-1 

3010  IF  I2>Log2n  THEN  3030 
3020  C( I2)-2a( 1 1  —  1 2  > 

3030  NEXT  12 


!  2A 1 0- 1 024 
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3040  K-l 

3030  FOR  11-1  TO  C< 10) 

3060  FOR  12-11  TO  C<9>  STEP  CU3/ 

3070  FOR  13*12  TO  C  3)  STEP  C<9> 

3080  FOR  14-13  TO  C<7>  STEP  C<8> 

3090  FOR  13-14  TO  C<6>  STEP  CC7) 

3100  FOR  16-13  TO  C<3)  STEP  C<6> 

3110  FOR  17-16  TO  C<4>  STEP  C<3> 

5120  FOR  18-17  TO  C<3>  STEP  C<4> 

3130  FOR  19-18  TO  C<2>  • TEP  C<3> 

3140  FOR  110-19  TO  C(l>  STEP  C<2> 

3130  J-I10 

3160  IF  K>J  THEN  3230 
3170  fi-X(K) 

3180  X<IO-X<  J> 

3190  X(J)-A 
3200  A-Y<K> 

3210  Y<IO-Y<  J> 

3220  Y< J)-A 
3230  K-KM 
3240  NEXT  110 
3230  NEXT  19 
3260  NEXT  10 
3270  .NEXT  17 
5280  NEXT  16 
3290  NEXT  IS 
3300  NEXT  14 
3310  NEXT  13 
3320  NEXT  12 
3330  NEXT  II 
3340  SUBEND 
3330  I 

3360  DEF  FNArg<  X, Y)  !  PRINCIPAL  *F0<2.' 

3370  IF  X-0  THEN  A-.3*PI*SGN<Y> 

3380  IF  XO0  THEN  A-ATN<Y/X> 

3390  IF  X<0  THEN  A-At>PHK  1~2*<Y<0>  ) 

5400  RETURN  ft 
3410  FNEND 
5420  f 

5430  SUB  D*t  *<N, X<  *  >  > 

3440  OPTION  BASE  1 
3450  REDIM  X<2,N) 

3460  DATA  1,2, 6, 3, 1,1, 2, 1,4, 3, 3, 2, 1,3, 6, 1,2, 4, 3, 9 
3470  DATA  2, 1,0, 1,3, 3,0, 1,2, 6, 2, 2, 4, 2, 3, 3. 6, 9, 8, 2 
3480  READ  X<*> 

3490  SUBEND 
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NUMBER  OF  DATA  POINTS  IN  EACH  PROCESS  N 
MAXIMUM  ORDER  OF  PREDICTIVE  FILTER  Pt»*x 
SIZE  OF  FFT  NFFt  ■  256 

PROCESS  NUMBER  1 

126311214532  1 

PROCESS  NUMBER  2 

2101530126224 


MEANS  OF  INPUT  DATA  <Av*;: 
3.35 
3.2 


COVARIANCE  MATRIX  OF  INPUT  DATA  <R>: 
5.7275  .83 

.83  6.16 


AKAIKE  INFORMATION  CRITERION: 
P  AiC<P> 

0  . 354363690E+0 1 

1  . 324 1 47349E+01 

2  . 3132O9545E+01 

3  . 342425965E+0 l 

4  . 235777S53E+01 

5  . 248589498E+0 1 

6  . 266 1 1 7185E+0 1 


Pb«st  «  4 


Ubcst : 

.54533344383  .226987553323 

.226387553923  4.00248888163 


FORWARD  PARTIAL  CORRELATION  COEFFICIENTS 
P  Ap< 1 , 1 >  flp  t  2,  1  > 

1  . 32684786 1 E+00  296O5502SE-0 

2  -.49707941 2E+88  . 85 ! 05533SE-0 1 

3  -. 172083146E+O0  . 237335358E+00 

4  -. 1436316S3E+00  -. 1 10686655E+00 

5  -. 51 6644665E-0 1  -. 44054O762E+O0 

6  . 98 1 306062E-O 1  -. 7366 16304E-O 1 


BACKWARD  PARTIAL  CORRELATION  COEFFICIENTS: 
P  Bp  <  1 , 1 >  Bp(2,l> 

1  . 391694481E+00  . 451665794E+00 

2  -.657848877E+00  . 178422353E+00 

3  -. 218767381E+00  . 685 3501 77E-01 

4  1286399O8E+00  .  10O617477E+01 

5  257137623E-01  .  1941242O6E  +  00 

6  . 182731463E+00  284463 163E+00 


■  20 

»  6 


5  6  1  2  4  8  9 


2  3  5  6  9  8  2 


i 


Ao1  t , 2  >  Ap<2, 2 > 

. 4  3350S 953E+00  . 492296101E+0O 

. 1 77974 150E+00  - . 2750 1 75 1 2E+00 

. 586 121 O47E-0 1  . 154503723E+0O 

. 7347436 2 9 E+00  1 247 1 5363E+00 

. 1 93329039E+O0  .  333560 1 94E+80 

-.2565S0155E+00  . 734752737E+00 


Bp' 1 . 2'  Bp<2, 2> 

1 3971 73 14E-01  . 42744943  IE  +  00 

. 271 4022 1 OE-O 1  1921435 1 2E+00 

. 1 77465 J55E+00  .  1 29759954E +00 

365594759E-01  - . 944 1 4 1 856E-0 1 

-.21 320 J  30SE-00  .  1  179205 1OE+00 

. 62622073 l E -01  . 161491679E+00 
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FORWARD 

PREDICTIVE  FILTER 

COEFFICIENTS  FOR 

Pb«»* : 

P 

Ap(l,  1) 

Ap<2, 1 > 

ApC 1 , 2  > 

Ap<.2, 2; 

1 

. 235726249E+0O 

. 174118 1 49E+00 

.2963703736+00 

•670170372E+00 

2 

. 600483360E+O0 

-.  164936772E+0O 

. 412699679E+00 

-.442413930E+0O 

3 

. 40436O932E+0O 

. 342089409E+00 

-.23374 3642E+00 

. 23 1 443573E+00 

4 

. 143631688E+00 

-.  U068663SE+00 

.7347436296+00 

-.  1247 13363E+0O 

NORMALIZED  CORRELATION 

MATRICES  <Rrt): 

DELAY 

AUTOll 

CR0SS21 

CR0SS12 

AUT022 

0 

. 100000000E+01 

. 139734996E+00 

.  139734996E+00 

. 1000000O8E+O1 

1 

. 3P9669762E+O0 

. 402437203E-0 1 

. 493230946E+00 

■438307O47E+00 

2 

-. 23473 1 303E+00 

. 912480910E-81 

. 51 109O860E+0O 

. 640872368E-O 1 

3 

372945922E+00 

.  193208143E+00 

.  1171071 13E+0O 

. 663O36633E-01 

4 

22394 1 70 IE-01 

. 876238663E-0 1 

.  190792716E+0O 

. 100976438E+O0 

3 

. 396697360E+00 

-. 89679 1 098E-0 1 

.  112701303E+0O 

. 122691973E+00 

6 

. 338388400E+00 

-. 836262322E-01 

-.9313S9127E-01 

. 1 8838673 1 E-0 1 

SPECTRAL  DENSITY  MATRIX  AND  COHERENCE,  FROM  ZERO  FREQUENCY  <BIN  1): 

BIN  AUTOll  AUT022  R£<CR0S$12>  IM<CR0$S12>  MAG  SO  COH  ARGUMENT 

1  .  363234E-01  .  1 3O480E+08  .843704E-01  .OOOOOOE+Ol  .98O117E+00  .0000O0E+01 

2  . 362462E-01  . 130293E+O0  .347113E-01  -.234607E-02  .980O64E+00  -.300468E-O1 

3  .360132E-01  . 1 29740E+00  .8423S9E-81  -.306939E-O2  .979904E+00  -.601103E-O1 

4  .336327E-01  . 128822E+00  .334493E-01  -.734340E-02  .979S34E+00  -.902894E-O1 

3  .331023E-O1  .  1 27349E+00  .823603E-01  -.996103E-02  . 979230E+00  -. 120360E+00 

6  .344301E-01  . 123934E+0O  .809811E-01  -.  122873E-01  . 973746E+O0  -. 130331E+O0 

7  . 336226E-01  . 123994E+00  .793277E-01  -.14S083E-81  .978U3E+O0  -.130892E+00 

3  . 326890E-81  .121731E+00  .774193E-01  -.166O73E-01  .977342E+00  -.2113UE+00 

9  .  316400E-01  . 1 1 9230E+00  .732793E-01  -.135704E-01  .976419E+00  -.241838E+00 

10  . 304882E-01  .116460E+J0  .729330E-01  -.203337E-01  .973331E+0O  -.272537E+00 

11  . 492476E-01  .U3474E+00  .704O63E-01  -1220443E-01  . 974O39E+00  -. 303429E+O0 

12  . 479333E-01  .  U8309E+00  .677369E-01  -.233426E-01  .972384E+00  -.334499E+00 

13  .463623E-01  . 107002E+00  .649405E-01  -.248774E-01  .978333E+00  -.363795E+0O 

14  .431507E-01  .103393E+00  .628731E-01  -.260303E-01  .963928E+00  -.397343E+00 

13  .437138E-01  .  100121E+00  .391473E-01  -.270676E-01  .96S688E+00  -.429181E+00 

••• 

116  .493634E-02  .327736E-02  .180088E-02  .366373E-02  .638329E+00  .111448E+01 

117  .464126E-02  .492382E-02  .198232E-02  .323433E-02  .616239E+00  .103904E+01 

118  . 436922E-02  .461992E-02  .198172E-02  .234179E-02  .3946346+00  .961831E+00 

119  .4135226-02  .433933E-02  .204244E-02  .248491E-02  .373913E+00  .882822E+00 

120  . 393482E-02  .413733E-02  .208809E-02  .213893E-02  .334331E+00  .8O1897E+O0 

121  .376422E-02  .394377E-02  .21241SE-02  .183948E-02  .336176E+00  .7190S3E+80 

122  . 362023E-02  .379O07E-02  .213073E-02  . 153236E-02  .3196O3E+00  .634308E+O0 

123  . 338027E-02  .363814E-02  .217033E-02  . 1323986-02  . 30433SE+OO  . 347727E+00 

124  .  34021  IE-02  . 335042E-02  .2183096-02  .  103104E-02  .492038E+00  .439427E+O0 

123  . 332403E-02  .346487E-02  .219333E-02  .33O527E-03  .431338E+O0  .369382E+00 

126  . 326463E-02  .339991E-02  .2202S9E-02  .629879E-03  .4729236+00  .273418E+00 

127  . 322293E-02  .333433E-02  .220767E-02  .413919E-03  .466828E+00  .136214E+00 

128  . 319821E-02  .332731E-02  .221037E-02  .286809E-03  .463144E+00  .932912E-01 

129  . 319001E-02  . 331836E-02  .221123E-02  .OOOOOOE+Ol  .461911E+08  .000000E+31 


TRAPEZOIDAL  SUMS  OF  SPECTRA: 

3.72678260464  6.  139933763S2  .330232934123 

COVARIANCES  OF  INPUT  DATA: 

3.7273  6.16  .83 
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ALIASED  NORMALIZED  CORRELATION  MATRICES: 


DELAY 

AUT011 

CR0SS21 

CR0SS12 

AUT022 

8 

. 180808808E+81 

. 139783721E>00 

. 1 3973372 1 E>00 

. 100000800E>O1 

1 

. 339662823E+00 

. 482483863E-8 1 

. 493316301E+O0 

. 43330731 OE+00 

2 

-. 23471 1073E>00 

.912137898E-01 

. 311 1 14131E+0O 

. 640949626E-O 1 

3 

-. 372888877E+88 

. 1931S3313E*0O 

. 1 1707440OE>00 

. 6631 4797 1 E-0 1 

4 

-.22382391  IE-81 

. 878380884E-0 1 

. 19078134 1E>O0 

. 1 00979 1 79E+O0 

3 

. 3988325 1 SE+90 

-.896403926E-01 

. 1 1 2729849E+O0 

. 1226341 13E+08 

6 

.  3303 1 063 l E>00 

-. 836812969E-0 1 

-. 951538121E-01 

. 138794710E-81 

7 

-■ 343828289E-81 

.609690818E-02 

-. 999 1 7932 6E -81 

-. 796893328E-02 

8 

308860393E*08 

. 82485 1 230E-8 1 

. 131798148E-01 

. 197442248E-0 1 

9 

282499 138E >08 

.629998356E-81 

. 176921946E>00 

-.  191 1 13963E-01 

18 

.  793873938E-01 

-. 238734388E-0 1 

. 10S767436E>00 

-.21 1019703E-O1 

1 1 

.318325251E+00 

-. 68984377SE-81 

-. 978171228E-01 

. 306843393E-02 

12 

.  192498120E  +  00 

-. 429532422E-81 

-. 1 43308 1 63E+00 

. 290497237E-0 1 

13 

123481 195E+00 

. 24 1 337936E-0 1 

-.432238S45E-01 

. 21269 t 423E-01 

14 

29438281 1008 

. 63864 1 473E-0 1 

. 104637982E>00 

-. 2 1934390 7E-01 

13 

Jfc  JL  JL 

1 28 1 S4922E+08 

. 382603975E-0 1 

. 1 2372330 1 E+OO 

-. 313103389E-01 

w  w  w 

118 

2441941 12E-02 

.913297310E-83 

. 313669734E-02 

-. 782794335E-03 

117 

.278370631E-02 

. 783865771E-03 

.  133633926E-02 

-.  620439684E-03 

118 

. 411244193E-02 

-. 122243230E-83 

-. 167993100E-O2 

.  1 77338396E-03 

119 

.  8t/3212018E-03 

-. 344466436E-03 

-.263339723E-02 

. 730513242E-03 

128 

-.280736 94 4E-92 

-. 822303928E-03 

-. 764928706E-03 

.  3 1 9056832E-03 

121 

-.  284788 177E-02 

-. 179269831E-84 

. 172370237E-02 

239248236E-03 

122 

.  448108016E-03 

. 889991232E-03 

.2 1896976  IE-02 

-. 71 999773 7E -03 

123 

. 2383350 1 3E-82 

. 191284639E-02 

. 329164628E-83 

-. 43932643  IE- 03 

124 

.  U2888107E-82 

.  992673477E-84 

16423626OE-02 

.  286993430E-83 

123 

-.  121802873E-82 

-. 183365279E-02 

-. 1 71669783E-02 

. 69861833OE-03 

128 

-. 131737804E-82 

-. 124028530E-02 

-.  491074133E-O4 

. 378392473E-03 

127 

. 30633S380E-03 

-.866418380E-84 

.  1 46460666E-02 

- .  3303741 32E-83 

123 

. 148247837E-82 

. 133638383E-82 

.  1 33630383E-02 

-.  6989723S2E-03 
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Appendix  B 

General  Filter  and  Spectral  Relations 

For  each  integer  k,  let  Hk  be  a  rectangular  matrix  of  (complex)  constants,  to  be 
called  the  filter  impulse  response  at  delay  kA,  where  A  is  the  time  sampling  in¬ 
crement.  The  number  of  filter  outputs  need  not  equal  the  number  of  filter  inputs. 
For  multichannel  filter  excitation  Wn  at  time  nA,  the  filter  output  at  time  nA  is  given 
by  the  discrete  convolution 


"n-k  • 

(B-l) 


where  the  summation  extends  over  all  nonzero  summands. 

For  a  stationary  excitation,  let  the  correlation  matrix  of  complex  input  process 
{Wn}  at  delay  l&bc 


W 

n 


(non-diagonal  matrix) , 


(3-2) 


where  the  overbar  denotes  an  ensemble  average,  and  the  superscript  H  denotes  a 
conjugate  transpose.  The  z-transform  of  input  correlation  sequence  {P^  }  is 


£(z)  s  A J  z’1  P,  > 

A 

and  the  spectrum  of  input  process  { Wn}  is,  for  real  frequency  f, 

Q(f)  *  (2.(exp(i27rfA))  ■  A  £  exp(-i2irfAA)  P. 


(B-3) 


■  a£  exp(-iui)  P 
A 


(B-4) 


where  we  let 


u  =  2trfA 


(B-5) 


The  correlation  matrix  of  the  filter  output  process  { Xn} ,  at  delay  l  A,  is  given  by 
using  (B-l)  and  (B-2): 


X 


n 


•  E 

k,m 


H, 


W 


n-k 


w"  , 

n-A-m 


»k 


P  .  HH 
A+m-k  m 


The  z-transform  of  output  correlation  sequence  { }is,  by  use  of  (B-6), 


(B-6) 


B-l 
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<J(z)  =  A£  z\  -4l  z'1  r  Hk  Pltn.k 

■L  ^  H,  AZ 

k  jc  in 

Now  define  the  z-transform  of  filter  sequence  {Hk}  by 

H(z)  -  £  z'k  H. 

k  K 

and  define  the  quantity 

H  H 

H  (z)  =  [H(z)]  ,  including  conjugation  of  z. 

Then 


We  now  employ  (B-8),  (B-3),  and  (B-10)  in  (B-7)  to  obtain 

G(z)  -  H(z)  £(z)  h“  (£)  . 

The  spectrum  of  output  process  {X„}  is  then,  at  real  frequency  f, 

G(f)  «  G(exp(i2Tt£A))  ■  A  £  exp(-i27rfA£)  Rff 

£ 

-  H(f)  Q(f)  H^f)  . 

where  we  employed  (B-l  1),  (B-4),  and  set 

H(f)  *  H(exp(i2trfA))  *  £  exp(-i2Tr£Ak)  H. 

k  K 

We  also  employed  the  property  that 

HH(f)  =  [H(f)]H  »  £  exp(i27tfAk)  H?  , 

k  K 


•  (B-7) 

(B-8) 

(B-9) 

.  (B-10) 

(B-U) 

(B-l  2) 

(B-l  3) 

(B-14) 


B-2 
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""(sp rraar)  ■  «H(«p(1*»a»  ■ «"(«  .  (B-15) 

Finally,  from  convolution  (B-l),  we  obtain  the  z-transform  of  output  data 
sequence  {Xn}  as 


X(2)  =  E  z’n  xn  "  » 

n  (B-16) 

where  we  used  (B-8)  and  defined 


0/(z)  -  £  z‘n  Wn 
n 


(B-17) 


The  major  results  thus  far  are  given  by  (B-l),  (B-16),  (B-l  1),  and  (B-12)  for  a 
general  filter  and  excitation. 

ARMA  Process 

For  a  multichannel,  autoregressive,  moving-average  process,  the  recursion  is 
given  by 


xn  ■  E  Ek  *„.k  -  E  fk  v* 


The  z-transform  of  this  equation  is 


(B-l  8) 


where 


X(z)  -  E(z)  X(z)  ♦  F(z)  W(z) 


E(z)  =  E  z'k  Et  ,  F(z)  =  E  z"k 


Then  we  can  solve  (B-19)  for  X  (z)  as 

X(z)  -  [I  -  E(z)]"1  F(z)  W(z)  -  H( z)  W(z) 
where  the  transfer  function  from  input  to  output  is 

H(z)  =  [I  -  E(z)]"1  F(z) 


(B-19) 


(B-20) 


(B-21) 


(B-22) 


in  terms  of  the  parameters  of  recursion  (B-l 8).  But  now  (B-22)  is  in  the  framework 
of  the  presentation  above;  namely,  the  spectrum  of  output  process  {Xn}  is,  from 
(B-12), 


G(f)  =  H(f)  Q(f)  ^(f)  , 


(B-23) 


B-3 
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where  - 

H(f)  -  [I  -  lit)]  F(f)  , 

E(f)  ■  E(exp(i27rfA))  *  J]  exp(-i2irfAk)  E,  , 

k  K 

F(f)  -  F(exp(i2irfA))  -  £  exp(-i2rrfAk)  F.  . 

k  K 


Example 

As  an  example  of  (B-18),  consider  the  multichannel  first-order  recursion 

X  ■  E.  X  .  ♦  W 
n  l  n-i  n 

with  the  input  spectrum  for  { Wn}, 

Q(f)  -  A  I  for  all  £  . 

This  is  a  white  process,  uncorrelated  from  channel  to  channel.  Then 

E(z)  ■  z"1  Ej  ,  F(z)  *  I  , 

H(z)  -  (I  -  z’1  E^"1  , 

G(£)  «  A  H(f)  !!“(£)  . 


Specialization  to  Two-Channel  Process 

We  further  specialize  example  (B-25)  to  the  two-input,  two-output 
process  characterized  by  coefficient  matrix 


Then  (B-27)  and  (B-22)  yield  transfer  function 


a  b 
c  d 


(complex  coefficients) 


B-4 


(B-24) 


(B-25) 


(B-26) 


(B-27) 


channel 


(B-28) 


TR  6533 


HU) 


1  -  z"1  a  -z'1  b 


•z_1  c  1  -  z"1  d 


-1 


1  -  z-1  (a  ♦  d)  ♦  z~  (ad  -  be) 


1  -  z"1  d 


z"1  e 


z-1  b 


1  -  z"1  a 


There  follows 


HH(1/z*)  -  [H(1/z*)]H 


- 5 - 

1  -  z(a  ♦  d)*  ♦  z  (ad  -  be)* 


1  -  zd*  zc* 


zb* 


1  -  za* 


and 


G(  z) 


d  H( z)  ^(l/z*)  = 


*nU) 

S21(z) 


g12(z) 

g22(z) 


l 

D 


1  ♦  | b | 2  ♦  | d| 2  -  z'1  d  -  zd* 
-ab*  -  cd*  +  z’1  c  +  zb* 


-a*  b  -  c*d  +  z”1  b  ♦ 
1  ♦  | a| 2  ♦  | c| 2  -  z"1 


where 

D  »  [l  -  z-1(a  ♦  d)  ♦  z"2(ad  -  be)]  [l  -  z(a  ♦  d)*  +  z2(ad  - 


Inspection  of  denominator  D  in  (B-32)  reveals  that  G(z)  has  poles  (i.e. 
elements  of  matrix  G  (z)  have  poles)  at 


(B-29) 


(B-30) 


zc* 

a  -  za* 

(B-31) 

be)*] 

(B-32) 

all  the 
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a  ♦  d 


a  ♦  d)  -  4(ad  -  be) 


and  at 


_1^ 

2* 


exp(i  arg(zj)  . 


(B-33) 


(B-34) 


That  is,  even  though  recursion  (B-25)  is  limited  to  first-order  regressive  and  white 
independent  excitations  (see  (B-26)),  G  (z)  has  four  poles  in  the  finite  z-plane. 
Generally,  if  tf(z)hasapoleatza,then  //(1/z*)  has  a  pole  at  l/z*;soif 


la>  9  T  exp(i0) ,  then  1/z*  »  exp(i0)/r 


(B-35) 


has  the  same  argument. 


In  addition,  element  gt  ((z)  in  (B-31)  has  a  zero  at »  and  three  zeros  in  the  finite 
z-plane,  at 


z 

0 


0  and  zQ 


1  *  ibi 2  - 


.  V(l  ♦  1  b  I  ^  ♦ 

2d* 


The  product  of  the  latter  two  zero  locations  is  d/d*  =  exp(i2arg(d)).  Thus,  the  auto 
spectrum  of  process  1  has  three  zeros  and  four  poles,  even  though  the  multichannel 
recursion,  (B-25),  is  first-order  regressive.  Similar  behavior  is  true  of  the  auto 
spectrum  of  the  second  process,  as  well  as  the  cross  spectrum  between  the  two 
processes. 

The  magnitude-squared  coherence  for  this  example  is,  from  (B-31)  and  (B-5), 
g12(exp(iu))  g21(exp(iu)) 

g11(exp(iu))  g22(exp(iu)) 

j  ■  2 

|-  a*b  -  c*d  ♦  b  exp(-iu)  ♦  c*  exp(iu)| 

[|  1  -  d  exp(-iu)  |  2  |  b|  2 j  1  -  aexp(-iu)|2  +  |c[2j 

(B-37) 


This  has  four  zeros  and  four  poles  in  the  finite  z-plane. 


B-6 


Numerical  Example 

We  now  specialize  (B-28)  to  example  (6)  in  the  main  text: 
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-  - 

m 

- 

a  b 

.85 

-.75 

m 

c  d 

.65 

.55 

m 

(B-38) 


The  four  poles  of  the  spectrum  G(z)  are  located  at 


z  ■  .7  ♦  i  .6819  ,  1/z*  ■  .7330  +  i  .7140  , 

(B-39) 

and  the  zeros  of  g,,(z)  ire  at 

zfl  ■  0  ,  3.0646,  .3263,  ®  . 

(B-40) 


The  zeros  of  g12(z)  are  located  instead  at 

zQ  »  0  ,  .8802,  -1.3109, 


(B-41) 


The  magnitude-squared  coherence  simplifies  to 

1.0634  -  .056  cos(u)  -  .975  cos(2u) 

[1.865  -  1.1  cos (u) ]  [2.145  -  1.7  cos(u)]  * 


This  example  was  used  frequently  in  the  main  body  of  this  report.  The  peak  value  is 
.9990128  at  u  *  .772564,  or  2fA  «  .245915. 
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Experimental  Comparison 
Of  Three  Multichannel 
Linear  Prediction 
Spectral  Estimators 


S.  L.  Marpl* 

A.  H.  Nuttall 

ABSTRACT 

Single-channel  spectral  estimators  based  on  linear  prediction 
techniques,  such  as  the  maximum-entropy  method,  have  been  shown 
to  often  provide  better  spectral  stability  and  resolution  than  standard 
FFT  procedures  for  short  data  sequences.  Based  on  this  improved 
performance,  a  multitude  of  multichannel  linear  prediction 
techniques  have  been  promoted  for  processing  multichannel  data 
sequences.  Three  of  these  are  examined  In  the  paper  a  multichannel 
generalization  of  the  single-channel  Burg  algorithm  by  Nuttall,  a 
maximum-entropy  type  of  algorithm  by  Morf,  Vieira,  Lee  and  Kailath, 
and  a  multichannel  extension  of  the  covariance  method  of  linear 
prediction  implemented  by  Marple.  For  purposes  of  experimental 
comparison,  various  two-channel  data  sets  were  processed  by  the 
three  methods  td  produce  the  two  autospectra,  the  magnitude- 
squared  coherence  and  the  coherence  phase  associated  with  each 
data  set.  A  possible  deleterious  effect  of  signal  ‘feed-across’  be¬ 
tween  autospectra  and  in  the  coherence  has  been  discovered  in  all 
three  methods.  The  phenomenon,  due  to  inexact  pole-zero  can¬ 
cellation,  is  especially  prominent  for  short  data  sequences.  Based  on 
the  multichannel  results  given  here,  the  Nuttall  method  generally 
produced  the  best  spectral  estimates. 
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PARAMETRIC  SPECTRAL  ANALYSIS 

Experimental  comparison  of  three  multichannel 
linear  prediction  spectral  estimators 

S.  Lawrence  Marple  Jr.  and  Albert  H.  Nuttall 

Indexing  term  Signal  processing 

Abstract:  Sinflxluuirwt  spectral  estimators  based  on  lineat  prediction  techniques,  such  u  the  maximum- 
entropy  method,  have  been  shown  to  often  provide  better  spectral  stability  and  resolution  than  standard 
FFT  procedures  for  short  data  sequences.  Based  on  this  improved  performance,  a  multitude  of  multichannel 
linear  prediction  techniques  have  been  promoted  for  processing  multichannel  data  sequences.  Three  of  these 
are  examined  in  the  paper:  a  multichannel  generalisation  of  the  single-channel  Burg  algorithm  by  Nuttall. 
a  maximum-entropy  type  of  algorithm  by  Morf,  Vieira,  Lee  and  KaBath,  and  a  multichannel  extension 
of  the  covariance  method  of  linear  prediction  implemented  by  Marple.  For  purposes  of  experimental  com¬ 
parison,  various  two-channel  data  sets  wars  processed  by  the  three  methods  to  produce  the  two  autospectra, 
the  magnitude-equated  coherence  and  the  coherence  phase  associated  with  each  data  set.  A  possible  deleteri¬ 
ous  effect  of  signal  'feed -aero sa'  between  autospectn  and  In  the  coherence  hat  been  discovered  in  all  three 
methods.  The  phenomenon,  due  to  inexact  pole-tero  cancellation,  is  especially  prominent  for  short  data 
sequences.  Based  on  the  multichannel  results  given  here,  the  Nuttall  method  generally  produced  the  best 


spectral  estimates. 

1  Introduction 

Multichannel  digital  signal  processing  is  being  used  in  an 
increasing  number  of  application  areas,  particularly  in  the 
sonar  and  geoteismic  communities.  Until  recently,  most 
multichannel  digital  signal  processing  was  based  on  fast 
Fourier  transform  (FFT)  methods.  The  success  of  unichannel 
high-resolution  spectral  estimation  techniques,  like  the  auto¬ 
regressive  or  so-called  maximum-entropy  methods,  has 
encouraged  researchers  to  develop  multichannel  extensions 
in  hope  of  obtaining  performance  improvements  for  multi¬ 
channel  applications  comparable  with  that  seen  in  unichannel 
applications.  The  multichannel  extensions  are  all  based  on 
linear  prediction  concepts,  since  a  linear  prediction  filter 
whitens  an  autoregressive  process. 

Three  multichannel  linear  prediction  spectrum  analysis 
algorithms  are  examined  in  this  paper.  They  are  the  multi¬ 
channel  generalisation  of  the  Burg  algorithm  developed  by 
Nuttall  [  1  -4] ,  a  multichannel  maximum-entropy  spectral 
estimate  by  Morf,  Vieira,  Lee  and  Kailath  (5,6],  and  a  multi¬ 
channel  generalisation  of  the  covariance  method  of  linear 
prediction  as  implemented  by  Marple  [7| .  All  three  algorithms 
make  estimates  of  the  multichannel  linear  prediction  coef¬ 
ficients,  from  which  the  multichannel  autoregressive  auto¬ 
spectra  and  cross-spectra  may  be  computed.  From  the  cross¬ 
spectra,  the  magnitude-squared  coherence  and  coherence 
phase  may  be  computed. 

There  has  been  no  previous  attempt  in  the  literature  to 
compare  and  characterise  the  various  multichannel  linear- 
prediction/autoregressive  spectral  estimators.  An  experimental 
approach  is  used  in  this  paper  to  empirically  characterise 
performance  with  respect  to  two  signal  classes.  One  class  is 
an  actual  two-channel  autoregressive  process.  The  other 
class  is  a  set  of  tones  (sinusoids)  imbedded  in  a  coloured 
noise  process.  An  analytic  approach  for  calculating  multi¬ 
channel  algorithm  performance  was  felt  to  be  mathematically 
intractable,  given  the  very  complex  analysis  that  was  required 
to  characterise  the  single-channel  autoregressive  spectral 
estimate  [8,9] 
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Relative  to  the  particular  multichannel  results  reported 
here,  the  Nuttall  algorithm  generally  produced  the  best  results. 
Best  in  this  case  means  with  less  frequency  estimation  bias 
and  variance  than  the  other  methods  when  tested  against 
autoregressive  and  tonal  processes.  During  the  testing,  a 
■feed-across’  effect  was  discovered  that  was  common  to 
ail  techniques.  Narrowband  components  that  should  be 
present  in  only  one  channel's  autospectra  were  found  coupled 
into  another  channel's  autospectral  estimate.  This  is  shown  to 
be  due  to  inexact  pole-zero  cancellation  in  the  autospectral 
estimate.  Conditions  that  would  cause  spectral  line  splitting  in 
the  single-channel  case  were  found  to  also  cause  splitting  in 
two  of  the  three  linear  prediction  techniques  considered  here. 

2  Summary  of  techniques 

This  Section  provides  an  overview  of  the  three  multichannel 
linear  prediction  methods  considered.  If  we  define  Xn  as 
the  vector  of  M  channel  samples  at  time  index  n 

r*.on 


|><Af)J 

from  a  stationary  zero-mean  multichannel  process,  then  the 
covariance  function  at  tune  lag  k  is  given  by 

Rk  -  E[XntkX"]  * 

The  symbol  H  denotes  Hermitian  transpose,  and  E  denotes 
statistical  expectation.  Define  the  forward  linear  prediction 
error  for  prediction  order  p  as 

Yk  -  f 

n  ■<> 

and  the  backward  linear  prediction  error  as 

p 

7  =  V  D(P>  V 

_  Lin  -'h-p  +  n 
n  »0 

where  f.-tSf”!?  and  are.  respectively,  the  forward  and 

backward  linear  prediction  coefficient  matrices  ot  dimension 
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M  x  M.  Note  that  A#'  ■  Bip)  »/  (the  identity  metrix)  by 
convention. 

Minimisation  of  the  forward  and  backward  linear  predic¬ 
tion  mean  square  errors  tr  {ElYpYi1] }» tr  \UP)  and 
tr  { E[ZkZ F] }-  tr  {Vp),  respectively,  (tr  denotes  the  matrix 
trace)  will  yield  coefficient  matrices  that  satisfy  the  block- 
Toeplitz  normal  equation 


is  the  power  spectral  density  matrix  in  the  two-channel  case 
Here  Gu(f)  and  Gn(f)  are  ^e  autospectra  of  channel  1  and 
channel  2,  respectively,  and  the  magnitude-squared  coherence 
(MSC)  is  given  by 

MSC  -  \Gli(f}\il{Gu(f)Gn(/)) 


l 

A\p>  ... 

...A& a  <‘l 

~R0  R i  .  • 
Rt  Rp  . . 

...  Rp’ 

B<» 

B?-\  ... 

...  B^  l 

_Rp  Rp.  i  .. 

....  R9 

0  0 
0  V, 


The  solution  to  this  normal  equation,  when  the  covariance 
matrices  (Rk)f  are  known,  is  provided  by  the  multichannel 
(vector)  Levinson-Wiggins-Robinson  (LWR)  algorithm.  Briefly, 
this  algorithm  relates  the  order  p  solution  to  the  order  p  —  1 
solution  according  to  the  recursions 


The  coherence  phase  spectrum  (CPS)  is  simply 
CPS  -  arg  (C„(/)] 

3  Unknown  covariance:  Measured  data 


A,  -  i 

K«0 

-  -A,(K,.,)" 

Bj?'  -  -A"(f/,.,)-' 

U,  -  (l-A'f'B?')U,.i 

V,  - 

Ai’>  -  A?'"  +  APB#?*  for  1  <  k  <  p  -  1 

Blp)  -  ft'"  for  1  <  *  <  p-  1 

with  initial  conditions 
U„  -  K0  «  R0 

/t?»  «  BP  -  /  for  0  <  k  <  p 

The  M  x  M  matrix  Ap  is  often  called  the  reflection  coefficient 
matrix. 

Based  on  the  linear  prediction  coefficients,  the  multi¬ 
channel  autoregressive  power  spectral  density  matrix  estimate 
may  be  shown  to  be  ( 1  j 

CA(r)  -  Ar(/t(r)]-'f/p(/t(l/r*)]-H 
*  G»(z) 

where  At  is  the  sample  interval,  —H  denotes  the  Hermitian 
transpose  of  the  inverse,  the  asterisk  denotes  complex  conju¬ 
gate.  and 

-4(z)  -  t  W" 

m  «0 
P 

B(z)  *  £ 

n  •© 

The  substitution  r  *  exp  (jlnfAt)  is  made,  and  GA  is 
evaluated  as  a  function  of  the  frequency  /.  With  this  substi¬ 
tution  having  been  made,  then 


Gif) 


G„cn  cn(f) 

Gulf)  G:J(A 


The  three  linear  prediction  algorithms  examined  in  this  paper 
are  concerned  with  the  situation  in  which  there  are  data 
samples  available,  but  no  covariance  values  are  known  at 
any  lap.  Assuming  N  vectors  X„  of  channel  samples  have 
been  collected,  then  the  following  squared-error  and  cross¬ 
error  covariance  estimates  may  be  formed: 


B, 


1 

N-p 


I  YkY? 


r. 


i 

N-p 


1  A- 1  A*-, 


1 

N-p 


s 

I 


YkZ 


H 

a-i 


The  Nuttall  extension  of  the  Burg  algorithm  to  the  multi¬ 
channel  case  makes  use  of  the  LWR  algorithm,  with  the 
exception  of  how  the  reflection  coefficient  Ap  is  computed, 
since  covariance  values  are  now  unavailable.  Using  the  error 
covariance  estimates.  Ap  is  obtained  as  the  solution  of  the 
bilinear  equation 

+  =  -  2G„ 

in  order  to  satisfy  a  weighted  arithmetic  mean  criterion 
between  the  forward  and  backward  squared  errors  Ep  and 
fp- 

The  so-called  maximum  entropy  algorithm  of  Morf  er  al. 
also  uses  the  LWR  algorithm,  but  the  reflection  coefficient 
Ap  is  computed  as  the  weighted  geometric  mean  of  the  for¬ 
ward  and  backward  squared  errors: 

A,  *  IE,)-U'GP 

The  superscript  notation  —  1/2  means  the  square-root  matrix 
of  the  inverse,  since  a  normaiised  form  of  the  LWR  algorithm 
is  used  by  the  authors  of  this  linear  prediction  method 

Instead  of  forcing  the  LWR  recursions  lo  hold  in  the 
given  data  case,  as  in  the  two  previous  algorithms,  one  could 
separately  minimise  the  forward  and  backward  squared  errors 


tr 


N 

y 


n  mP+  I 
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a  ai  a  2  ai  a*  as  s  01  02  02  0.4  os 

B  traction  or  eantQte  frequency  d  fraction  el  (ample  frequency 


Fi*.  2  Autotpectra  and  coharance  of  20  ortrlappad  data  salt:  NuttaU ■  Burt  algorithm  / enaraliution 

a  G„  lutoepectrum 
t>  C„  eutoipectrum 
c  Meptitude-equered  coherence 
d  Ctiherence  pheee 


0  01  0  2  03  04  OS  0  at  02  03  04  as 

S  fraction  of  sample  frequency  n  traction  of  lampfe  frequency 


Fin.  3  Autospectra  and  coherence  of  20  overlapped  data  sett  \lorf  eta I  maximum -entropv  generalisation 

a  CM  autotptetrum 
b  (J,,  iutospcctrum 
c  M»(mttide«quued  coherence 
J  Coherence  phue 
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peak)  than  the  other  methods.  In  fict,  18  out  of  20  of  the 
phase  estimates  were  close  to  the  truth  for  the  Nuttail  algor¬ 
ithm,  whereas  none  of  those  given  by  the  other  techniques 
was  correct  at  all.  Note  that  the  four  plots  of  Fig.  4,  based  on 
the  forward  linear  prediction  estimate,  differ  little  from  those 
in  Fig.  S,  based  on  the  backward  linear  prediction  estimate, 
even  though  the  covariance  method  does  not  guarantee  that 
the  two  cases  will  yield  the  same  result. 

This  first  test  has  very  important  implications.  All  three 
techniques  should  work  well  against  a  simple  first-order 
autoregressive  process,  since  this  is  the  data  model  appropriate 
for  linear  prediction  spectral  estimators.  Clearly,  however, 
the  Nuttail  technique  works  best,  at  least  for  short  data 
records.  The  other  two  techniques  exhibit  more  variance 
than  is  typically  found  in  single -channel  versions  of  these 
methods.  For  a  multichannel  pth -order  regression  and  indepen¬ 
dent  white  excitations,  it  may  easily  be  shown  that  the  auto- 
and  cross-spectra  of  the  processes  each  possess  4 p  poles  and 
3 p  zeros  in  the  finite  r-plane  (of  which  p  zeros  occur  at  the 
origin).  This  is  in  contrast  to  the  single-channel  case,  where 
only  2 p  poles  (and  a  pth-order  zero  only  at  the  origin)  can 
occur.  Thus,  while  a  single -channel  linear  prediction  requires 
estimation  of  only  p  parameters,  an  M channel  approximation 
requires  estimation  of  M*p  parameters  per  channel.  For  a 
fixed  number  N  of  data  points  from  each  process,  the  esti¬ 
mation  of  an  increased  number  of  parameters  can  only  be 
done  with  increased  variance  since  the  total  number  of  data 
points  is  only  MN.  This  is  a  partial  explanation  of  the  results 


seen  in  Figs.  2-5. 

The  next  test  consisted  of  data  sequences  with  three 
sinusoids  in  a  coloured  noise  process.  In  channel  1,  sinusoids 
with  fractions  of  sampling  frequency  of  0  1,  0.2  and  0.24 
were  used.  The  respective  amplitudes  were  0.1,  1  and  1, 
while  the  respective  initial  phases  were  0,  90  and  235°  The 
coloured  noise  process,  which  had  most  of  its  power  above 
the  ’  frequencies  of  the  sinusoids,  was  generated  by  passing 
white  noise  of  0.05  variance  through  a  digital  filter  with  a 
raised-cosine  spectral  response  between  0.2  and  0.5  of  the 
sampling  rate.  In  channel  2,  sinuso’ds  of  fractional  frequencies 
0.1,  0.2  and  0.4,  amplitudes  of  0.1, 1  and  l.and  initial  phases 
of  0,  210  and  25*  were  generated.  A  coloured  noise  process 
similar  to  channel  1,  but  independently  generated,  was  added 
to  the  sinusoids.  Note  that  the  sinusoid  components  at  frac¬ 
tional  frequencies  0.1  and  0.2  are  common  to  both  channels. 
A  data  sequence  of  64  points  was  generated  for  each  channel. 
Figs.  6-8  depict  the  autospectra  and  coherence  magnitude 
and  phase  for  order  p  »  12  estimates  by  the  three  linear 
prediction  methods.  Note  that  in  all  plots,  some  energy  of 
the  0.24  fractional  frequency  sinusoid,  present  only  in  channel 
1.  has  coupled  into  the  Gn  estimate  for  channel  2,  and. 
conversely,  the  0.4  fractional  frequency  sinusoid,  present  only 
in  channel  2,  has  coupled  into  the  (7U  estimate  for  channel 
1.  The  MSC  plots  also  show  spikes  at  the  tone  frequencies 
of  0.24  and  0.4,  where  ideally  the  coherence  should  be  zero. 
This  ‘feed-across’  artifact  is  an  undesirable  effect:  however, 
it  appears  unavoidable  whenever  processing  short  data 
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sequences  with  multichannel  algorithms.  If  the  data  sequences 
from  each  channel  are  processed  separately  by  the  single¬ 
channel  Burg  linear  prediction  algorithm,  the  autospectra 
shown  in  Fig.  9  are  the  result.  Note  that  these  spectra  do 
not  give  a  false  tonal  indication.  By  increasing  the  data 
sequence  length  to  500  samples,  there  is  an  improvement  in 
the  feed-across  effect  in  the  multichannel  autospectra  in  Fig. 
10,  although  the  coherence  still  has  strong  sharp  spikes.  Thus 
it  seems  that  the  attractive  properties  of  single-channel  linear 
prediction  spectrum  analysis  relative  to  periodogram  spectrum 
analysis  when  processing  short  data  records  do  not  extend  to 
the  multichannel  case  without  problems.  The  feed-across 
effect  leads  to  false  indications  of  narrowband  components, 
especially  when  processing  short  data  sequences.  A  detailed 
analysis  of  the  cause  of  the  feed-across  effect  is  presented  in 
the  following  Section. 

The  final  test  case  examined  the  spectral  line  splitting  pheno¬ 
menon  that  has  been  reported  in  the  literature  for  single¬ 
channel  linear  prediction  spectrum  analysis.  A  data  sequence 
of  101  samples  of  a  single  tone  at  0.0725  fractior  of  sampling 
frequency,  unit  amplitude  and  43°  initial  phase,  added  to  the 
same  coloured  noise  process  (variance  10'°)  as  in  the  last 
test,  was  generated  for  channel  1.  A  comparable  101 -sample 
sequence  was  generated  for  channel  2,  but  the  tone  was  placed 
at  a  fractional  frequency  of  0.2175.  Both  cases  are  known  to 
cause  line  splitting  in  the  single-channel  Burg  algorithm. 
Figs.  11-13  show  the  autospectra  generated  by  the  three 
multichannel  linear  prediction  methods  for  order  p  *■  15. 
Line  splitting  is  present  in  the  Nuttall  and  Morf  methods, 
but  not  in  the  covariance  method.  Thus  line  splitting  be¬ 
haviour  carries  over  into  some  of  the  multichannel  techniques. 


5  Properties  of  linear  predictive  spectral  analysis 
techniques 

The  behaviour  of  the  linear  predictive  techniques  in  the 
presence  of  tonals  is  of  importance  to  anyone  using  these 
spectral  analysis  procedures.  Wt  now  present  some  of  the 
important  properties  of  multichannel  linear  prediction 
techniques  and  illustrate  these  properties  by  an  example 
from  the  Nuttall  algorithm  for  the  two-channel  auto¬ 
regressive  process  in  eqn.  1 ,  with  a  tone  added  solely  to  the 
channel-2  process.  The  tone  strength  is  —  6  6  dB  relative  to 
the  sample  power  in  the  second  component  of  Xn,  and  the 
tone  is  located  at  fractional  frequency /e  »  0J;  this  is  basically 
the  example  considered  in  Fig.  1 1  of  Reference  4 

The  autospectra  and  coherence  estimates  for  a  data  run 
with  N  ■  1000  data  points  and  p  *  8  are  given  in  Fig.  14. 
There  is  the  desired  strong  tonal  indication  at  fc  in  GZJ , 
but,  in  addition,  there  is  a  weak  undesired  contribution 
at  fc  in  £7,, .  Since  this  tone  was  never  added  to  process  1, 
this  indication  in  (7,,  has  leaked  across'  in  the  mathematical 
data  manipulations  of  the  linear  predictive  algorithm.  This 
leakage  is  unavoidable  and  will  be  present  in  all  multichannel 
linear  predictive  procedures;  it  is  due  to  the  fact  that  we 
must  work  with  finite  data  sets,  thereby  resulting  in  slightly 
inaccurate  filter  coefficients.  The  effects  on  the  poles  and 
zeros  of  the  autospectral  and  coherence  estimates  are  discussed 
below.  The  MSC  estimate  in  Fig.  14c  has  developed  two 
notches  and  a  sharp  spike  near  frequency  fe,  while  the  phase 
in  Fig.  14 d  has  gone  through  an  abrupt  2ir  change  in  that 
neighbourhood. 

In  order  to  explain  the  various  behaviours  of  the  spectral 
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Pi*.  9  Autosnutre  of  sinusoids  in  noise  (64  samples I  Single -channel 
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estimates  in  Fig.  14,  we  need  to  develop,  in  more  detail, 
the  matrix  spectral  estimate  (see  eqn.  F-16  of  Reference  1) 

C(t)  »  [A(z)\-'Vp[A(Uz-)\-h  (2) 


where 

/«(z)  -  /+  £  *’”AP 

«■  I 


(3) 


For  real  data  and  filters,  U„  and  {4^*}  are  real.  For  a  two- 
channel  application,  eqn.  3  can  be  expressed  as 


/l  (i) 


[*„  U)  ^»ta U) 

>zi(z)  *::(?) 

where  {hkl(;)}are  scalars  of  the  form 


A*i(z)  -  I  z' "“«>/('») 

R«0 


There  follows  for  eqn.  2 
C(z) 


hJ2(l  lz)  — h:,(l/r) 
-A,j(l/z)  A„<l/z) 


(4) 


(5) 


1 

hii(z)  Ajj(z) 

“ll 

“u 

D(t)D[\!z) 

-Aji(z)  A„(r) 

“51 

“jj 

(6) 


where 


a  (ttl  aulospvctrum 
b  c‘„  iutosp«ctrum 


D(:)  =  A, ,(:)/!;, 


(7) 
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We  denote 


A*i(*)  -  *‘pd»t(*)  (g) 

hki(  1/*)  “ 

where  ?»j(z)  and  r»j(z)  are  polynomials  of  degree  p  in  z 
with  real  coefficients.  Then 

D(z)-f'>Q(z)  j  (9) 

£>(l/z)  -  R(z)  j 

where  (2(a)  and  R(z)  are  polynomials  of  degree  2p  in  z.  it 
follows  that 


G(0)  -  <»ii(P)«n(P)-«isO>>»ii(P) 

R(0)  -  1 

neither  of  which  is  zero;  thus  Q(z)  and  R(z)  have  no  zeros 
at  the  origin  of  the  z-plane.  The  spectral  estimate  can  now 
be  expressed  as 


C(Z)  “  G(*)«(a) 

where  the  2  x  2  matrix  in 


JV„(z)  Afu(z) 
Af„(z)  A /„(z) 
eqn.  10  is  given  by 


(10) 


<f:a(*)  -<7u(*) 

«n  “u 

Fjl(2) 

-  ?»(*)  <Jii  (2) 

“11  “H 

,-/n(2) 

F.l(2) 

(ID 


All  the  functions  in  eqn.  10  and  expr.  11  are  polynomials 
inz. 

For  the  real  data  and  filters  employed  here.  the.  2 p  zeros 
of  Q(z)  occur  in  conjugate  pairs  (or  in  real  pairs)  Furthermore, 
they  are  all  inside  the  unit  circle  Ct  in  the  complex  z-plane, 
this  property  was  proved  in  eqn.  33  of  Reference  3  Thus  we 
only  need  to  search  the  interior  of  the  unit  hemisphere  in  the 
upper-half  z-plane  for  p  zeros  of  Q(z). 

Furthermore,  if  Q(z)  has  a  zero  at  z0,  then  R(z)  has  a  zero 
at  l/z0;  thus  G(z)  in  eqn  10  has  4 p  poles  in  the  z-plane. 
2 p  in  the  upper  halfplane,  of  which  p  are  inside  the  unit 
hemisphere.  A  typical  4-tuple  of  poles  in  G(z)  is  given  by 
z0,  zj,  l/zo,  1/zS;  there  are  p  such  4-tuples.  These  poles  are 
common  to  all  the  auto-  and  cross-spectral  estimates  in  eqn. 
10. 

Similarly,  a  typical  term,  Nkt  (z),  in  the  numerator  of  C(z) 
in  eqn.  10  is  a  polynomial  of  degree  2p  in  z,  of  which  p 
zeros  will  be  located  in  the  upper-half  z-plane.  Thus  eqn.  10 
shows  that  (every  term  of)  G(z)  has  p  zeros  at  0,  p  zeros  at 
••  and  Ip  zeros  in  the  finite  plane.  We  need  only  search  for 
the  p  zeros  of  Nh ((z)  in  the  upper-half  plane,  for  k.  I  *  1 , 2. 

In  order  to  explain  the  behaviour  of  the  coherence  estimates 
given  by  the  linear  predictive  techniques,  we  start  with  the 
complex  coherence  at  frequency  f  as  given  (for  At  =  1 )  by 


C(f) 


N_  »  fexp  (/2  m/)) _ 

(exp(/'2ir/))yVn(exp(/2jr/))| 1,5 


(12) 


The  squared  coherence,  generalised  to  the  entire  z-plane,  is 


V'iz) 


jV.I  (£) 

Nu(z)Nll(z) 


I  Af„(r)Af„(z)\ 
P1  ,Vlt  (z)zV„  (Z)j 


(13) 
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The  following  are  general  properties  of  the  jVfc,(z)  polynomials: 

(а)  If  /V„  (z)  his  a  zero  at  r,  exp O'0i).  it  also  has  a  zero  at 
(1/r, )  exp  (/0t). 

(б)  If  A^jj  (z)  has  a  zero  at  r,  exp  (jd  t ),  it  also  has  a  zero  at 
(l/'i)  exp  (/$,). 

(c)  If  fVl2(z)  has  a  zero  at  r0  exp  (jBa),N ,,  (z)  has  a  zero  at 
(l/r0)  exp  (/«#). 

In  fact,  property  (c)  is  a  special  case  of  the  genetal  rule  that 

/V„(z)  -  z,pJVi,(l/z)  for  all z  (14) 

Now  we  return  to  the  example  of  Fig.  14,  where  a  tone  was 
present  only  in  the  channel-2  data,  at  frequency  Since 
process  1  has  no  tone  at/t,  JVu(z)  develops  two  zeros  near  ze  3 
exp  (/2ir/«)  -  exp  (J ir0.6),  one  inside  the  unit  circle  C, 
and  one  outside  Ct ,  tending  to  cancel  the  one  zero  of  D(z) 
inside  C,  and  the  one  zero  of  0(1  /z)  outside  C i ,  which  are 
near  zt,  so  that  the  autospectral  estimate  Gtl(f)  is  well 
behaved  near  fc.  However,  the  cancellation  is  not  perfect, 
and  the  small  spike  at  ft  in  Fig.  14a  remains.  For  this  example, 
the  pole  location  inside  Ct  is  0.993591  exp  (/»  0.600104), 
whereas  the  zero  location  inside  Ct  is  0.992697  exp  (/> 
0.600015).  Since  the  pole  is  closer  to  C(,  a  positive-going 
perturbation  occurs  in  Fig.  14a  at  ft. 

Similarly,  the  cross-spectrum  of  <7(z)  ideally  should  have 
no  indication  at  fe  sines  there  is  no  tone  in  process  1 .  In  order 
to  counter  the  zeros  of  D(z)  and  0(  1  /z)  near  ze,  ,V(1  (z) 
develops  rwo  zeros  near  zc.  For  the  same  example,  they  are 
at  0.996958  exp  (jt rO.592593)  and  0.983192  exp  (jit 
0.615533),  both  of  which  happen  to  be  inside  C( .  Other 


examples  have  shown  that  these  two  zeros  can  both  be  o  side 
Ct ,  or  one  can  be  inside  and  one  outside  C, 

Finally,  since  process  2  does  have  a  tone  at/c,  A'22  if)  does 
not  develop  any  zeros  near  zc.  Thus  the  zeros  of  0(z)  and 
0(1 /z)  dominate  near  ze,  and  the  estimate  Gn(J)  in  Fig  146 
develops  a  large  value  near/c,  as  desired 

The  squared  coherence  estimate  in  eqn  13  is  independent 
of  0(z)  or  Q(z).  However,  recording  to  the  discussion  above, 
it  has  two  double-zeros  mar  ze,  owing  to  the  /V,22  (z)  term 
These  four  zeros  can  lie  either  all  inside  C, ,  all  outside  C, , 
or  two  inside  and  two  outside  C|.'^l(z)  also  has  two  poles 
near  zc,  owing  to  the  two  zeros  of  iV„  (z)  near  zc  one  pole 
lies  inside  Cl ,  the  other  lies  outside  C\ .  Since  there  are  no 
poles  and  zeros  near  ze  that  cancel  exactly,  some  very  fine 
detail  can  develop  in  the  coherence  estimate  in  the  neighbour¬ 
hood  of  ze.  Sharp  notches  and  spikes  are  the  rule,  not  the 
exception,  in  the  MSC  evaluated  on  C|  in  the  neighbourhood 
of  a  tonal  frequency  c  wing  to  this  imperfect  cancellation  of 
poles  and  zeros.  The  phase  variations  can  be  so  rapid  that 
large  FFT  sizes  are  required  to  track  it  accurately  The  two 
double-zeros  of  /V^f z)  cause  this  rapid  variation,  especially 
if  they  are  all  located  on  the  same  side  of  C, ;  the  two  zeros 
of  iV,,  (z)  are  always  located  on  opposite  sides  of  C,  and  so 
do  not  themselves  lead  to  a  very  rapid  phase  change  near  zc, 
although  they  greatly  influence  the  MSC  in  that  region.  In 
general,  the  squared  coherence  2  (z)  has  Ip  double-zeros 
and  4 p  poles  in  the  complex  z-plane.  none  restricted  to  be 
inside  or  outside  of  C[ ,  however,  they  all  occur  in  conjugate 
pairs. 

Since  order  p  »  8  used  in  this  example  is  larger  than 
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FI*.  13  Autospectra  of  data  set  that  produced  lint  split  ting  in  Fits.  1 1 
and  12:  Marpit  covariance  algorithm  generalisation  ! forward j 
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necessary  to  account  for  a  single  tone  and  a  narrowband 
spectral  component,  the  extra  poles  [zeros  of  0(z)|  are 
distributed  fairly  uniformly  inside  C\ ,  with  radii  in  the  range 
0.5  to  0.7.  The  extra  zeros  of  polynomials  <V*,(z)  can  lie 
anywhere,  either  inside  or  outside  of  C, 

6  Discussion  and  conclusions 

Experimental  results  for  two-channel  linear  prediction  auto- 
and  cross-spectral  estimation  have  been  presented  for  a 
first-order  autoregressive  process  and  for  cases  with  inter¬ 
fering  tones.  It  has  been  shown  that  some  misleading  estimates 
may  be  obtained  because  of  feed-across  in  the  mathematical 
manipulations  of  the  finite  lengths  of  data  from  each  channel. 
This  feed-across  manifests  itself  as  narrow  spurious  spikes 
in  the  spectral  and  coherence  estimates.  In  order  to  circumvent 
this  problem,  while  maintaining  the  high-resolution  properties 
of  linear  prediction  techniques,  the  following  philosophy 
for  multichannel  spectrum  analysis  is  suggested. 

Suppose  we  are  given  finite  data  records  of  three  station¬ 
ary  processes  x(r),  y(t)  and  z(r),  and  we  wish  to  estimate 
all  the  autospectra  and  cross-spectra  involved.  The  Blackman 
and  Tukey  and  weighted-FFT  approaches  evaluate  the  auto¬ 
spectrum  of  each  process  separately.  Thus  the  spectrum  of 
x(f)  is  estimated  without  interference  from  y(f)  and  r(f); 
the  availability  of  the  data  records  for  y(t)  and  z(r)  plays  no 
part  in  the  eventual  autospectral  estimate  for  x(t).  Ad¬ 
ditionally,  the  cross-spectral  estimate  for  processes  xU)  and 
y(t)  is  independent  of  the  available  data  on  the  z(r)  process. 
Finally,  the  coherence  estimate  between  two  processes  is 
independent  of  any  additional  data  records  for  other  (statisti¬ 
cally  related)  processes. 


Fi*.  14  Autospectra  and  coherence  of  first-order  autoregressive  process  plus  single  sinusoid  H000  samples,  p  =  $1  'JuttailBurg  algorithm 
generalisation 
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On  the  other  hand,  the  three  multichannel  linear  predictive 
spectral  analysis  approaches  give  autospectral  estimates  of  the 
x(f)  process  that  are  dependent  on  the  available  values  of  y(t) 
and  z(f).  Also,  the  cross-spectral  estimate  between  x(f)  and 
y(r)  is  dependent  on  the  particular  z(t)  data  available.  This 
procedure  can  be  poor  for  short  data  lengths  if,  for  example, 
y(t )  contains  a  strong  tone  at  /0  that  is  not  present  in  x(t) 
or  z(f).  Thus  estimates  of  spectra  £»„(/),  Gxy(J)  and  GX,(J) 
all  contain  tonal  indications  at  f0  that  should  not  be  there. 
These  spurious  tonal  indications  are  due  to  feed-across 
between  the  available  finite  data  segments  of  the  various 
processes. 

This  raises  the  following  questions: 

(i)  Should  the  estimate  of  Gxx{f)  be  determined  only  from 
the  available  x(f)  data  record? 

(ii)  Should  the  estimate  of  Gxy(J)  be  determined  only  from 
the  available  x(t)  and  y(t)  data  records? 

(iii) lf  coherence  Cxy(fa)  m  0,  why  use  y(.)  to  estimate 

Gxx(f0)1 

(iv) lf  coherence  Cxy(f0)  “  1,  why  use  the  completely 
statistically  dependent  y(t)  data  to  estimate  Gxx(f0)1 

This  philosophy  of  discarding  ‘irrelevant’  data  would  be 
consistent  with  the  Blackman  and  Tukey  and  FFT  approaches. 
Carrying  this  philosophy  on,  we  are  led  to  the  following: 
estimate  Gxx(j)  solely  from  the  x(r)  data  by  some  single¬ 
channel  linear  predictive  technique.  Then  estimate  cross¬ 
spectrum  <7„(/)  or  coherence  C(f)  directly,  by  some  linear 
predictive  technique  whose  sole  goal  is  linear  prediction 
of  x(t)  from  y(t)  and  vice  versa,  with  no  interest  in  or  diver¬ 
sion  from  simultaneous  estimation  of  Gxx(f)  or  Gyy(f). 
By  this  means,  we  can  concentrate  on  extracting  all  the 
relevant  cross-spectral  information  with  maximum  stability 


and  resolution.  Other  cross-spectra  of  interest  between  particu¬ 
lar  pairs  of  available  processes  can  be  similarly  obtained, 
one  at  a  time. 
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ABSTRACT 

Estimation  of  multi-channel  auto-spectra,  cross -spectra,  and  coherences 
via  linear-predictive  techniques  suffers  from  cross-feed  between  the  limited 
number  of  data  points  available  of  each  channel  process.  Here,  the  coherence 
between  two  channels  Is  estimated  directly,  without  diversion  from  auto¬ 
spectra  estimation,  so  as  to  minimize  only  the  valid  computable  linear- 
predictive  errors  that  can  be  formed  from  given  finite  data  records. 
Furthermore,  the  least-squares  minimization  procedure  Is  constrained  so  as  to 
yield  physical ly-consi stent  magnitude-squared  coherence  estimates,  and  can  be 
accomplished  by  a  fast  algorithm  In  a  recursive  fashion.  The  resulting  non¬ 
linear  programming  problem  that  Is  encountered  Is  solvable  via  a  recursion  em¬ 
ploying  the  Hessian  of  an  unconstrained  error  measure.  Examples  of  applica¬ 
tion  of  the  technique  to  short  data  records  yields  good  coherence  estimates. 


INTRODUCTION 

Estimation  of  the  magnitude-squared  coherence  (MSC)  between  two  stationary 
processes  x  and  y  has  usually  been  accomplished  by  estimating  the  auto-spectra 
and  cross -spectrum  of  the  two  processes  and  forming  the  ratio 


)g,ff)r 

Here,  f  Is  frequency,  Gxx(f)  and  Gyy(f)  are  the  auto  spectral  estimates, 
and  tiXy ( f )  is  the  cross  spectrum  estimate. 

Recently,  a  more  direct  method  of  estimating  the  MSC  was  presented  (1,2), 
whereby  two  filters  are  used  to  linearly  estimate  y  from  x  and  also  x  from  y. 
The  product  of  the  two  filter  transfer  functions  is  then  used  as  an  estimate 
of  the  MSC.  In  this  way,  no  denominator  estimates  (as  in  Eq.  1)  are  ever  re¬ 
quired.  Some  related  work  Is  given  in  (3). 

Hov/ever,  there  are  some  limitations  in  this  past  work  that  need  further 
consideration.  In  (1,  Eq.  4),  the  squared  errors  were  not  confined  to  the 
valid  errors  that  can  be  computed  from  given  finite  data  records  of  x  and  y. 
This  results  in  the  claim  (1,  page  647)  that  the  matrix  that  must  be  inverted 
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Is  Toeplltz.  Although  this  is  true,  the  use  of  invalid  error  terms  and 
Toeplitz  matrices  has  been  found  in  (4)  to  yield  poorer  auto-spectral  esti¬ 
mates,  and  probably  does  so  for  MSC  estimation  as  well.  The  adaptive  approach  ‘ 

in  (2,  Eq.  6)  uses  only  available  data,  but  does  not  make  optimum  use  of  given 
limited  data  segments,  in  terms  of  variance  reduction. 

f 

Iri  addition.  In  both  (1)  and  (2),  since  no  constraints  are  imposed  on  the  r 

two  filters  that  do  the  linear  predictions  of  y  from  x,  and  x  from  y,  respec-  t 

tlvely,  the  product  of  the  transfer  functions  need  not  be  real,  although  the  \ 

true  (unknown)  MSC  Is  always  real.  Thus,  in  (1,  Eq.  14),  the  magnitude  of  the  , 

product  is  taken,  whereas  In  (2,  Eq.  7a),  no  mention  is  made  of  the  complex 
nature  of  the  right-hand  side.  The  reason  for  this  behavior  is  the  independ¬ 
ent  calculation  of  the  two  linear-predictive  filters  from  finite  data  records.  !> 

Finally,  in  both  (1,  Eqs.  4  and  13)  and  (2,  Eq.  2),  the  number  of  forward 
weights  is  set  equal  to  the  number  of  backward  weights.  This  unnecessary  lim¬ 
itation  should  be  eliminated,  as  may  be  readily  seen  by  considering  the  case 
where  one  process  is  a  (noisy)  time-delayed  version  of  the  other.  Insistence 
on  an  equal  number  of  forward  and  backward  weights  can  result  In  the  unneces¬ 
sary  computation  of  many  near-zero  weights. 

■ 

In  this  paper,  we  limit  the  least-squares  calculation  solely  to  the  valid  ' 

errors  that  can  be  formed  from  finite  data  records;  we  constrain  the  two  fil¬ 
ters  that  do  the  linear  predictions  so  as  to  yield  a  real  MSC  estimate;  and  we 
allow  the  number  of  forward  and  backward  weights  to  be  arbitrary.  We  also  use 
a  fast  recursive  algorithm  to  solve  for  the  filter  coefficients. 


BASI S  OF  TECHNI  QUE 

For  continuous-time  stationary  processes  x  and  y,  the  linear  non-causal 
filter  with  input  x,  that  minimizes  the  mean-square  error  in  predicting  the 
value  of  y  at  the  same  time  instant,  has  a  transfer  function  given  by 
GyX(f )/Gxx(f ) .  Conversely,  linear  prediction  of  x,  based  upon  data  record 
y,  is  accomplished  by  a  linear  filter  with  transfer  function  GXy(f )/Gyy( f ) . 

The  product  of  these  two  transfer  functions  is  then  precisely  tne  MSCbetween 
x  and  y. 

For  discrete-time  stationary  processes  x  and  y,  exactly  the  same  relations 
ensue,  except  that  the  spectra  are  discrete  Fourier  transforms  of  discrete 
correlation  functions.  Again,  the  two  filters  are  allowed  to  be  non-causal 
and  of  infinite  extent  in  the  Impulse  response  domain. 

In  the  practical  case,  the  various  required  auto-  and  cross-correlations 
and  spectra  above  are  not  known,  only  finite  data  records  are  available,  and 
the  filters  must  have  limited  impulse  response  durations.  Also,  the  mean- 
square  (ensemble  average)  error  must  be  replaced  by  a  measurable  physical  ly- 
meaningful  sum  of  squared  errors  and  then  minimized  by  choice  of  the  free 
parameters.  The  details  of  such  a  procedure  to  accomplish  MSC  estimation  are 
presented  below. 
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FORMATION  OF  VALID  ERRORS 


Suppose  that  data  records  of  x(m)  for  Mj  <_  m  <  M2  and  of  y(n)  for  Nj 
<  n  <  N?  are  available.  We  linearly  estimate  y(n7  from  the  available  data 
record  \x(m)}  according  to 

(j(n)2  jpElaOi  x(n-k);  Eq‘  2 

where  p  and  q  are  arbitrary  Integers.  We  select  -q  <  p  without  loss  of  gener¬ 
ality;  l.e.,  p+q  >_  0.  These  Integers  should  be  chosen  so  that  the  region  of 
{x(mj\  ,  on  which  y(n)  Is  statistically  dependent.  Is  covered  by  (non-causal) 
Impulse  response  fa(k)}  .  Similarly,  estimation  of  x(m)  is  accomplished 
according  to  the  linear  filtering  operation 

x(.»)s  2l  Uk)  yM).  Eq- 3 

k*-r 

Eqs.  2  and  3  are  consistent  in  their  selection  of  the  extents  of  the  non¬ 
zero  coefficients  In  Impulse  responses  (a(k)}  and  fb(k))  ,  In  that  the  statis¬ 
tically  dependent  portions  of  x  and  y  are  covered  equally  well,  regardless  of 
which  process  Is  predicted  from  the  other. 

In  order  to  compare  y(n)  with  y(n),  and  X(m)  with  x(m),  and  thereby  form 
valid  errors,  we  must  restrict  n  and  m  according  to 

Nj  s  *  s  Nlf ,  wVe/e  f\)^  »  w*ax(Ni>M,+-p),  min  E1-  4 

MjSWiM*,  u>U*  Mj*  wox(m,/N,+^)>  M,=  »''' (MxA"  p)j  Eq-  5 

where  it  is  presumed  that  N3  £  N4  and  M3  <  M4.  The  sums  of  squared 
errors  are  then 


s  |vj(v»)-^a(k)x(n-k)  Eq.  6 

~  *.«N,  J  n-Nj  1 

2  U)-£uk)y(»-k)2.  Eq*  7 


The  data  points  required  on  the  right-hand  sides  of  Eqs.  6  and  7  are  all 
available  in  the  given  records;  i.e.,  these  equations  involve  only  valid 
errors. 
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UNCONSTRAINED  LEAST-SQUARES  SOLUTIONS 

The  set  of  optimum  filter  weights  (a0(k)}  that  minimize  error  Ey  Is 
given  by  the  solution  of 


k= 

where 


ftnr(i,k)  s  i!Lx(w-jOx*(rM)  -fir  E 9 

0)  *  jW  /(»-■?)  hr  Eq.  10 

Eq.  8  constitutes  p+q+1  linear  equations  In  the  p+q+1  unknowns  £a0( k - 
Furthermore,  with  an  appropriate  shift  of  time  origin,  a  recursive  fast  algo¬ 
rithm  (5)  exists  for  the  minimization  of  Ey  In  Eq.  6,  i.e.,  the  solution  of 
Eq.  8.  Similarly,  the  Independent  minimization  of  Ex  In  Eq.  7,  by  choice  of 
optimum  filter  weights  \b0(k)]  ,  can  be  realized  by  a  second  call  to  that 
same  fast  algorithm.  Thus,  we  have  an  efficient  procedure  for  determining  the 
optimum  unconstrained  filter  weights  that  separately  minimize  Ex  and  Ey; 
these  are  called  the  unconstrained  least-squares  filters. 

We  can  now  evaluate  transfer  functions,  and  form  the  product  A0( f ) B0( f ), 

$  *  2,  ^  Ik)  (■>  2t rf-k  ),  Bo  5  i-  Wk)  **?(-*  2rf  k), 

ir-t  *s'r 

as  an  approximation  to  the  MSC  for  |fl  <  1/2.  An  example  of  the  result  for 
1000  coninon  data  points  In  each  sequence  and  for  p  s  q  *  6  is  given  in  Fig. 

1.  The  actual  data  points  were  generated  by  a  two-channel  first-order 
auto-regressive  process  according  to  (6,  pages  5-6) 

x(M  =  jr  x(k-i)  -.if  3  Ow)  +  u(k), 

uM  ..(S-xOt-i)  -h.  5Ttj(k-l)  +•  yfk^  Eq.  12 

where  ju(k)}  and  £v(kj]  are  Independent  white  uniformly-distributed  random 
processes. 

The  real  and  imaginary  parts  of  A0(f)B0(f)  are  plotted  as  solid  curves 
in  Fig.  1,  while  the  true  MSC  for  Eq.  12  Is  plotted  as  a  dashed!  curve.  Since 
errors  Ex  and  Ey  were  Independently  minimized,  the  imaginary  part  of 
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Ao(f)B0(f)  is  not  zero,  which  Is  not  consistent  with  a  MSC  which  must  be 
real.  However,  It  Is  conceivable  that  the  size  of  the  Imaginary  part  of 
Ao(f)B0(f)  could  serve  as  an  Indicator  of  the  statistical  accuracy  of  the 
real  part  as  an  estimator  of  the  MSC,  If  p  and  q  are  large  enough;  this  possl 
blllty  has  not  been  pursued. 


CONSTRAINED  LEAST-SQUARES  SOLUTIONS 

In  order  that  the  transfer  functions  In  Eq.  11  have  a  real  product  for  all 
f.  It  may  be  shown  (for  real  filter  weights)  that  the  convolution  of  filter 
weight  sequences  {a(k)\  and  [b(k)}  must  be  even.  Explicitly,  this  means  that 
we  must  have  (for  p+q  1) 

l(k)  k&i-K)  -fer  |<ii<p+£.  Eq.  13 

This  equation  constitutes  p+q  quadratic  equality  constraints  on  the  2(p+q+l) 
unknown  filter  coefficients. 

We  now  must  form  a  single  physlcally-meanlngful  error  quantity  and  mini¬ 
mize  It  subject  to  the  p+q  quadratic  constraints  In  Eq.  13.  We  select  the 
average  of  the  two-squared  errors  defined  In  Eqs.  6  and  7;  this  seems  reason¬ 
able  when  the  two  available  sequences  are  normalized  to  have  the  same  energy. 
Mathematically,  this  translates  Into  determining  the  stationary  points  (not 
minima)  of  the  quantity 

Q  *  i(Ex+  Ej")  1-  -21a(k) -  k(-"-K)],  Eq.  14 

where  {x(n|are  Lagrange  multipliers;  we  set  {a(k)}  and  {b(k)}  equal  to  zero 
except  for  the  ranges  utilized  above  In  Eqs.  2  and  3. 

The  solution  for  the  stationary  points  of  Eq.  14,  obtained  by  setting  all 
the  partial  derivatives  of  Q  with  respect  to  Ia(k)]  ,  $b(k)}  ,  and  [x(n)}  equal 
to  zero,  results  in  3(p+q)+2  non-linear  equations.  Ordinarily,  the  numerical 
solution  to  these  equations  would  be  very  difficult;  in  fact,  there  are 
generally  many  solutions.  However,  we  already  have  a  good  approximation  (at 
least  for  many  data  points)  to  these  solutions,  in  the  form  of  the  solutions 
to  the  unconstrained  least-squares  approach  of  the  previous  section.  Further¬ 
more,  recall  that  the  unconstrained  least-squares  filters  can  be  determined  by 
a  fast  algorithm.  Thus,  if  we  use  a  recursive  technique  and  employ  the 
Hessian  (7)  of  the  quantity  Q  in  Eq.  14,  along  with  the  starting  values  from 
the  unconstrained  solutions  and  zero  starting  values  for  the  Lagrange  multi¬ 
pliers,  we  will  have  an  efficient  recursive  method  of  getting  the  desired 
result,  namely  a  purely  real  MSC  estimate  via  minimization  of  a  physically- 
meaningful  error  criterion. 

The  results  of  this  constrained  procedure  for  the  same  pair  of  1000 
data-point  sequences  as  above,  and  for  p  =  q  =  6,  is  shown  in  Fig.  2,  after 
the  recursion  for  the  stationary  point  of  Q  has  reached  its  final  values.  The 
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Imaginary  part  Is  zero,  and  the  real  part  Is  substantially  the  same  as  Fig.  1, 
except  for  a  slightly  better  behavior  for  the  small  MSC  values  near  the 
Nyqulst  frequency. 
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If  p  and  q  are  not  chosen  large  enough  to  encompass  the  regions  of  linear 
dependence  of  processes  x  and  y,  poor  estimates  of  the  MSC  can  result,  even 
though  the  number  of  data  points  gets  large.  On  the  other  hand,  too  large 
values  of  p  and  q  results  In  excessive  statistical  fluctuations  In  the  MSC 
estimate.  Thus,  the  familiar  trade-off  between  frequency  resolution  and  sta¬ 
bility  must  be  faced. 

For  many  cases,  the  real  part  of  A0{f)Bg(f)  for  the  unconstrained 
approach  may  suffice  as  the  MSC  estimate,  without  bothering  to  resort  to  the 
recursive  approach  for  a  zero  Imaginary  part.  This  would  also  avoid  the  need 
for  Inverting  a  large  non-ToeplItz  matrix  of  size  3(p+q)+2  In  the  case  when 
p+q  Is  large.  If  so,  use  of  a  fast  algorithm,  as  Indicated  here,  affords  a 
very  efficient  procedure  for  getting  good  MSC  estimates  via  mlnlmlzaUcn  of 
valid  errors  only. 
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A  TWO-PARAMETER  CLASS  OF  BESSEL  WEIGHTINGS  WITH  CONTROLLABLE 
SIDELOBE  BEHAVIOR  FOR  LINEAR,  PLANAR-CIRCULAR,  AND  VOLUMETRIC- 
SPHERICAL  ARRAYS;  THE  IDEAL  WEIGHTING-PATTERN  PAIRS 

INTRODUCTION 

A  wide  variety  of  time-domain  weighting s  for  spectral  analysis,  whose 
frequency-domain  windows  have  very  good  sldelobe  behavior,  have  been  presented 
In  [1,2].  Since  the  basic  mathematics  descriolng  the  response  of  a  weignted 
linear  array  can  also  be  written  as  a  Fourier  transform,  these 
velght i rig -window  pairs  have  Immediate  application  to  one-dimensional  array 
processing  as  well  as  spectral  analysis. 

Most  of  the  weighting- window  pairs  In  [1,2]  have  no  parameters  In  their 
design  equations;  that  is,  the  windows  are  fixed  and  cannot  be  altered,  as  for 
example.  In  the  Hanning  and  Hamming  windows.  A  few  windows,  such  as  the 
Dolph-Chebyshev  and  Kal  ser-Bessel  [3,4],  do  have  a  single  parameter  In  their 
design  equations  that  allows  for  a  tradeoff  between  the  mainlobe  width  and  tne 
ratio  of  mainlobe  to  peak  sldelobe.  However,  neither  have  any  control  over 
the  rate  of  decay  of  the  sldelobes,  the  Dolph-Chebyshev  case  having  no  decay, 
and  the  Kai  ser-Bessel  case  a  6  dB/octave  decay.  It  is  obvious  that  in  order 
to  control  both  the  si delobe  decay  and  the  mai  nlobe-to-Deak-  si  delobe  ratio,  a 
two-parameter  family  of  weightings  Is  necessary.  And  It  Is  desirable 
(although  not  necessary)  for  the  window  to  possess  a  simple  analytical  form 
that  can  be  easily  understood  and  evaluated  for  a  range  of  parameter  values. 
Such  a  class  of  Be ssel  weightings  i  s  pre  sented  in  thi  s  report. 

For  the  array  application,  the  weighting  is  applied  as  a  multiplicative 
factor  in  the  spatial  domain;  the  response  to  plane  wave  arrivals  from  various 
directions  Is  called  the  pattern,  rather  than  the  window.  Here  we  will  give  a 
tw-parameter  family  of  weighting-pattern  pairs  for  use  with  arrays  in  one, 
two,  or  three  dimensions,  and  shall  Indicate  the  ideal  weightings  and 
corre qjondi ng  patterns  in  all  cases.  Special  cases  of  this  family  will  be 
shown  to  Include  some  of  the  weightings  that  are  currently  employed  in  array 
and  signal  processing. 
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RESPONSES  OF  ONE-,  TWO-,  AND  THREE-DIMENSIONAL  ARRAYS 


LINEAR  ARRAY 


We  consider  a  continuous  line  array  located  on  the  x-axis  in  the  range 
( -R , R )  and  subject  to  symnetrlc  weighting  wj(x)  for  |xl  <  R.  For  a 
single-frequency  plane  wave  of  wavelength  x,  arriving  at  angle  da  relative 
to  the  normal  to  the  line  array,  the  array  voltage  response,  by  use  of 
time-delay  steering  to  look-angle  is 

R 

g(u)  =  J'  dx  exp  £l2*y(  sinda-s1n^  )J  w^x).  (1) 

-R 

By  letting  s  *  x/R  and  by  using  the  symmetry  of  weighting  w^t  *e  can  express 
response  (1)  as 


g(u) 


r  ZV 

d si—  )  cos(  us)  w(  s) , 
C 


vWiere  normalized  weighting 


U) 


w(  s)  *  (2*/*R  WjlRs) , 
and  dimensionless  parameter 


(3) 


(4) 


incorporates  the  relevant  features  of  array  geometry,  look  angle,  and  the 
arrival  wavelength  and  angle. 


Thus  the  response  (2)  of  a  line  array  is  a  cosine  transform  of  the 
normalized  weighting.  As  an  example,  rectangular  weighting  yields  response 
g(u)  prooortional  to  sin  u/u,  which  has  its  first  few  nulls  at  u  =  it,  2*,  3». 
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PLANAR  ARRAY 

The  voltage  reqaonse  of  a  continuous  planar-circular  array  of  radius  R,  to 
a  plane  wave  of  wavelength  \  arriving  at  (polar,  azimuthal)  angles  (fta>  $a)  t 
and  subject  to  weighting  which  depends  only  on  the  distance  from  the  center  of 
the  array,  Is  derived  In  appendix  A,  culminating  in  the  result  (A-ll).  It  is 


ds  s  JQ(us)  w(  s) , 


where  w(  s)  1  s  the  normalized  weighting  and 


u  s  2*5.  £s1n2^  +  s1n2ba-  2s1n^sim6a  cos^ -ea^J  .  (6) 

Here  (^,  e^)  are  the  (polar,  azimuthal)  look  angles;  that  is,  the  response 
(5)  of  a  planar-circular  array  i  sAatt zero-th  order  Bessel  transform  of  the 
normalized  weighting.  As  an  example,  rectangular  weighting  w  yields  response 
g(u)  oroportlonal  to  Ji(u)/u,  which  has  Its  first  few  nulls  at  u  !  3.B3, 

7.02,  10.17. 


VOLUMETRIC  ARRAY 

The  voltage  regionse  of  a  continuous  vol umetrlc- spherical  array  of  radius 
R,  with  weighting  dependent  only  on  the  distance  from  the  center  of  the  array, 
is  derived  In  appendix  B.  The  result  Is  given  by  (B-10)  in  the  form 


g(u)  -  w(s). 


where  now 


u  r  2t 


^2- 2c o s^ c o s/ba  -  2sin^sinda  cos(e^-ea)j 


The  other  parameters  are  as  explained  in  the  previous  subsection. 
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Equation  (7)  has  the  basic  form  of  a  sine  transform.  As  an  example, 
rectangular  weighting  w  yields  response  g(u)  proportional  to 

sin  u-u  cos  u 

I?  ’ 

which  has  Its  first  few  nulls  at  u  *  4.49,  7.73,  10.90. 

UNIFIED  FORM  FOR  ARRAY  RESPONSES  IN  DIFFERENT  DIMENSIONS 


The  results  In  (2),  (5),  and  (7)  for  the  array  voltage  response  In  one, 
two,  and  three  dimensions,  respectively,  appear  to  be  quite  different. 
However,  they  can  all  be  written  In  the  basic  form 


1 

g(u)  =  ds  K{ u,  s)  w(  s),  (10) 

0 

where  the  kernel 


K(u,  s) 


J  (us) 
u 


cos(u  s) 

s  tyus) 

,  sln(us) 
u 


j 


(11) 


Here  i  s  a  Bessel  function  of  order  i,  and  we  have  used  [5;  10.1.1, 

10.1.11,  10.1.12].  Thus  all  the  req>onses  are  basically  Bessel  transforms  of 
the  normalized  weighting  w,  with  the  corre spondence s  given  in  the  following 
table. 


Table  1.  Identification  of  Values  of  m  in  (11) 
Nunber  of  Dimensions  Value  of  u 


4 


1 

2 

3 


-1/2 

0 

1/2 
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If  we  substitute  (11)  In  (10),  we  have  the  explicit  result  for  the 
re^jonse  pattern: 


1 

g(u)  -  J'  ds  s/jjj  J(us)  w(s). 
0  u 


(12) 


Inspection  of  the  properties  of  the  Bessel  function  reveals  that  g(u)  as  given 
by  (12)  Is  even  In  u;  see  [5;  9.1.10].  Thus  we  only  need  to  Investigate  g(u) 
for  u  >  0. 
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HANKEL  TRANSFORM  PAIRS 


The  Hankel  transform  pair  of  order  u  Is  given  by  [6;  page  136] 

<C 

f(u)  *  f  ds  (us/*  J  (us)  F(s)  for  u  >  0, 

J  u 

o 

«o 


F(s) 


I 


du 


{  su)*  J  (  su) 

M 


f(u)  for  s  >  0. 


(13) 


Thus  knowledge  of  either  function  f  or  F  for  positive  arguments  enables 
determination  of  the  other  function  by  an  Integral  transform.  Under  the 
Identification 


F(s) 

*  s  vt  s),  f(u)  *  u  g(u), 

(14) 

(13)  takes  the  form 

r 

g(u)  =  J 

ds  s J(u$)  w(  s)  for  u  >  0, 

(lb) 

0 

06 

w(s)  = 

J 

du  ug)  J  (  su)  g(u)  for  s  >  0. 

W 

(16) 

0 

Equation  (15)  1  s  more  general  than  (12)  in  that  it  allows  for 

weight i ng 

w(  s)  to  be  nonzero  for  s  > 

1.  Equation  (16)  1 s  complementary  in 

that,  given 

desired  pattern  g(u).  It  Indicates  what  weighting  w(  s)  is  required  for  s  >  d. 
However,  If  we  attempt  to  specify  some  desirable  pattern  g(u),  and  then  solve 
(16)  for  the  required  weighting  w(s),  it  will  generally  turn  out  that  the 
resultant  w(  s)  will  be  nonzero  for  s  >  1.  Thus  not  jin^  pattern  g(u)  can  be 
selected  if  we  Insist  on  a  finite- support  weighting  w<  s) ;  rather,  desirable 
candidate  patterns  can  be  substituted  in  (16)  and  the  corre spondi  ng  weighting 
vX  s)  evaluated  to  see  If  it  is  zero  for  s  >  1.  If  not,  the  candidate  pattern 
Is  disallowed  and  must  be  modified  or  discarded!  We  will  use  precisely  this 
procedure  in  a  later  section  when  we  determine  the  weightings  tnat  realize  tne 
ideal  patterns  in  various  nunbers  of  dimensions. 

*  If  pattern  g( u)  yields  w(  s)  =0  for  s  >  a,  the  scaled  pattern  g(u/a)  yields 

o.  2 

a  modified  weighting  a^  v/as),  which  is  zero  for  s  >  1,  as  desired. 
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DEFINITION  OF  TWO  BESSEL  FUNCTION  RATIOS 

It  will  be  very  convenient  notatlonally  In  the  following  to  employ  the 
shorthand  notations 


J-(z) 


_1 

2° 


2  k 

(-Z  /4)  , 

IciHa+l+k) 


OO  O  k 

j.  S'  !z  /4>  » 

2°  £*J  kl  P(  a+l+k) 


(17) 


(18) 


for  these  two  Bessel  function  ratios;  these  types  of  functions  have  already 
been  encountered  In  (11),  (12),  (15),  (16).  They  are  extensions  of  the 
lkn(z)  functions  dl scussed  In  [7;  page  56].  Both  functions,  ^a(z)  and 
&<  z),  are  single-valued  and  are  entire  In  z  for  any  a,  as  well  as  being 
entire  In  a  for  any  z  [5;  6.1.3,  9.1.1].  Special  cases  of  these  functions 
that  are  useful  here  are  given  by  [5;  9.6.6,  10.1.1,  10.1.11,  10.1.12, 
10.2.13,  10.2.14] 


(0)  = - - - 

a  2aH  a+1 ) 


i4a(oi 

<H0(z>  •  I0(  z),  d2x<  z)  .iiifi  .efjU)  -  zlj(z), 

^<z>’(#coShz. 


tf5/2(z) 


,(>  i  \  Z? 2  cosh  Z  -  sinh  z  ,0  ,  ,  (2]  ,  .  .  .  . 

C<3/2( z)  =[j]  - 3 - ,  MC_3/2(z)  *[-J  (z  sinh  z  -  cosh  z)  , 


$ 


3/2' 


y L  9 

(3+z  )  sinh  z  -  3z  cosh  z 

- 5 - 


& 


-5/2 


(z) 


€ 


[(3+z  )  cosh  z  -  3z  sinh  zj. 


(19) 


A  useful  prooerty,  which  is  obvious  from  (17)  and  (18)  and  which  will  find 
frequent  application  here,  is 
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0  (+1z)  *l$(z), 

Q  a  a 


lX  (±1z)  (z). 

a  v  a 


(20) 


These  relations  hold  true  for  all  values  of  z,  real  or  complex.  Properties 
similar  to  (19)  can  be  obtained  for  the  ^a(z)  functions;  for  example. 


_l/2(z)  “^.^(ilz)  cosh(+1z)  =(^j  cos(z). 


Other  useful  properties  follow  from  the  use  of  [5;  9.1.30  and  9.6.28], 

X(I)  *  -4^U)' 

and  from  [5;  9.1.27  and  9.6.26], 

^a(2)  '  7lk-2<Zl  ' 

|a(2»  --^[l^12'  *  2U-ll^-l'2]' 


(21) 
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A  CLASS  OF  BESSEL  WEIGHTINGS 


The  class  of  Bessel  weightings  that  we  are  suggesting,  regardless  of  the 
number  of  dimensions  of  the  array,  Is  given  by 


where  v  >  -I,  B  >  0, 


(22) 


and  w(  s)  *  0  for  s  >  1.  There  are  two  real  parameters  In  this  class,  namely, 
v  and  B.  For  the  special  case  of  v  *  0,  this  weighting  Is  already  known  as 
Kai ser-Be ssel  [3,4];  Its  pattern  has  nearly  the  optimum  energy  content  within 
a  specified  bandwidth  In  the  one-dimensional  application  to  spectral  analysis. 


Substitution  of  (22)  In  (12)  or  (15)  yields  the  closed  form  for  the 
pattern  [6;  page  30,  fourth  Integral,  and  5;  9.6.3] 


Here  we  have  employed  definitions  (17)  and  (18)  and  used  (20).  The  condition 
on  u  guarantees  convergence  of  Integral  (12)  at  s  *  0,  whereas  the  condition 
on  v  guarantees  convergence  of  Integral  (12)  at  s  =  1.  The  first  form  of 
(23B)  1  s  more  convenient  computationally  for  u  z  B,  whereas  the  second  form  is 
more  convenient  for  B  £  u. 


Welghtl nq-pattern  pair  ( 22) -(23)  are  the  fundamental  results  for  the  class 
of  Bessel  weightings  under  consideration.  They  apply  to  one-,  two-,  or 

three-dimensional  arrays  when  u  Is  specialized  to  -  -j,  0,  or  ^  respectively, 
and  vrfien  u  Is  interpreted  as  (4),  (6),  or  (8),  respectively.  The  parameter  B 
is  nornegative  and  will  be  snowi  to  control  the  ratio  of  mainloDe  to  peak 
sidelobe.  For  the  case  of  one  dimension,  u  =  the  kernel  of  transform  (23A) 
is  a  cosine  (  see  the  top  1  i  ne  of  ( 11) ) ;  i  n  thi  s  speci  al  c  a  se, 

tne  pattern  was  al  s0  independently  and  simultaneously  discovered 

by  Roy  Streit  [8], 
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WEIGHTING  CHARACTERISTICS 

I 

Weighting  (22)  Is  positive  for  0  £  s  <  1  since  v  >  -1  and  B  2  0;  see 
[5;  9.6.10].  In  addition.  It  Is  zero  and  therefore  continuous  at  s  *  1  if 
v  >  0.  In  fact  [5;  9.6.7], 

I 

(l-s)w 

pTvTl )  as  s~*1'*  (24) 

For  v  *  0,  weighting  w(  s)  1  s  monotonlcal  1y  decreasing  in  s  on  (0,1);  see  ! 

appendix  C.  However,  for  -1  <  v  <  0,  s)  possesses  an  Integrable  singularity 
at  s  *  1. 

Examples  of  weighting  (22)  are  plotted  in  figures  1-6  for  v  *  2,  1.5,  1, 

.5,  0,  -.5,  respectively.  Figure  5,  for  v  *  0,  corresponds  to  the  < 

Kal ser-Be ssel  weighting  [3].  For  the  larger  values  of  v,  the  weighting  blends  l 

snoothly  to  zero  at  s  *  1,  but  for  the  snail er  values  of  v,  the  behavior  of 
w(  s)  1  s  more  Irregular  at  s  *  1,  being  dl  scontinuous  for  v  *  0  and  infinite  ‘ 

for  v  <  0.  The  larger  values  of  B  lead  to  snoother  functions  that  are 
Gaussian- like;  in  fact,  for  s  <  1  [5;  9./.1J, 

v4s)~  eX^J—  exp  (-  4b^s^)  asB-*0®. 

(2*B]fV  V  2“  / 


More  generally. 


if 
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:ed  Weighting  for 


Normalized  Weighting  for 


Figure  5.  Normalized  Weighting  for  v 


Figure  6.  Normalized  Weighting  for 
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RESPONSE  fATTERN  CHARACTERISTICS 
The  pattern  was  given  by  (23)  as 

9(u)  (28, 

where  the  composite  order  of  the  Bessel  function  1  s 

a  *  u  +  v  +  1  .  (29) 

The  asymptotic  behavior  of  the  pattern  (28)  for  large  u  Is  available  from  (17) 
and  [5;  9.2.1]: 


hi  co slu  -  iaw  - 

g(ul~(v)  ' 


as  u-*»o. 


(30) 


Since  g  Is  proportional  to  the  array  voltage  response,  (30)  corresponds  to  a 


decay-  6a  +  3  dB/octave  as  u-»«. 


(31) 


Expressed  In  terms  of  the  original  dimension-parameter  u  and  weighting- 
parameter  v,  this  Is,  from  (29), 


decay  —  5u  +  6v  +  9  dB/octave  as  u  — 

[6v  +  6  dB/octave  for  one  dimension 
=  j6v  +  9  dB/octave  for  two  dimensions 
(6v  +  12  dB/octave  for  three  dimensions 


} 


(32) 


Thus  the  greater  the  number  of  dimensions,  the  faster  is  the  rate  of  decay  of 
the  sidelobes  of  the  re^onse,  for  a  common  value  of  weighting-parameter  v. 
Each  additional  dimension  adds  a  3  dB/octave  decay,  for  a  fixed  v. 
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Special  cases  of  the  pattern  (28)  are  available  upon  use  of  (19)  and  (20); 
for  example, 


g(u)  * 


All  of  these  relations  are  valid  for  all  u,  whether  u  Is  larger  or  smaller 
than  B;  of  course,  the  former  form  In  each  line  Is  more  convenient  for  u  <  B, 
vrt»11e  the  latter  1  s  more  convenient  for  u  i  3.  The  third  result  In  (33),  for 

a  »  -j,  Includes  the  pattern  In  one  dimension  (u  *  -  -jj,  v  =0)  for  the 
Kal  ser-Be  ssel  weighting  Iq^b^Ts^)  for  0  <  s  <  1. 


The  special  case  of  a  *  -  in  (33)  deserves  extra  attention;  thi  s  case 
will  be  called  the  Ideal  pattern: 


The  plot  in  figure  7  reveals  that  the  sidelobes  are  all  equal,  and  that  the 
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voltage  ratio 


Figure  7.  Ideal  Pattern  g i ( u ) 


mainlobe  level 
side  lobe  level 


cosh(B). 


(35) 


The  mainlobe  width,  as  measured  to  the  point  where  the  mainlobe  first  decays 
to  the  eventual  sidelobe  level,  is 


mainlobe  width  =  B . 


(36) 


The  abscissa  u  is  given  by  (4),  (6),  or  (8)  for  one,  two,  or  three  dimensions, 
re gjectively.  Determination  of  the  required  weighting  to  realize  the  ideal 
pattern  (34)  in  different  numbers  of  dimensions  is  taken  up  in  a  later  section. 
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If  a  <  -  -jj,  the  pattern  (28)  nas  Increasing  sidelobes  as  u  Increases;  see 

(30).  Therefore,  a  i  -  are  the  only  cases  of  practical  Interest  for  pattern 

(28). 

Plots  of  pattern  (28)  are  given  In  dB  In  figures  8-13  for  a  =  2,  1.5,  1, 
.5,  0,  -.5,  respectively,  for  various  values  of  B.  The  program  Is  listed  In 
appendix  D.  The  larger  values  of  a  realize  the  more  rapid  decay  of  sidelobes, 
but,  on  the  other  hand,  have  wider  malnlobes.  Figures  8-13  Indicate  the 
necessary  tradeoffs  between  mainlobe  width,  sldelobe  decay,  and 
malnlobe-to-sldelobe  ratio  that  must  be  considered  In  any  weighting  selection. 

A  small  chart  In  the  upper  right  quadrant  of  each  figure  indicates  some 
allowable  values  of  u  and  v  that  app'y  to  that  figure.  For  example,  in  figure 
8,  the  pattern  for  a  *  2  applies  to  all  the  following: 

1  3 

u  *  -  (one  dimension)  with  v  *  7  » 

or 

u  *  0  (two  dimensions)  wl th  v  *  1, 
or 

u  *  (three  dimensions)  with  v  «  .  (37) 

When  we  come  to  figure  11,  for  a  =  .5,  however,  the  case  of  u  =  -j,  v  =  -1  has 
an  asterisk  next  to  the  v  *  -1  entry.  The  reason  for  this  is  that  the 
Integral  (23)  leading  to  pattern  g(u)  wa s  convergent  only  for  v  >  -1,  and  now 
we  are  trying  to  violate  that  condition.  A  similar  cautionary  note  is 
indicated  in  figures  12  and  13;  in  fact,  all  three  cases  in  figure  13  violate 
the  condition  v  >  -1.  De^ite  this  preclusion,  we  shall  find  later  that  the 
required  weighting  does,  in  fact,  have  the  form  (22)  for  the  corre spondi ng  v 
values  given  in  figures  11  -  13,  but  requires  generalized  functions  with  a 
singularity  at  the  endpoint  s  =  1  of  the  interval.  This  extension  to  v  <  -1 
is  desirable  and  important  because  realization  of  the  ideal  pattern  (figures  7 
and  13)  requires  values  for  v  in  th i  s  region,  regardless  of  the  number  of 
dimension  s. 
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FIRST  NULL  LOCATION 


Let  zq  be  the  smallest  nonzero  null  location  of  Ja(z);  l.e. , 

Ja(za)  *  0. 


(38) 


A  short  list  of  {za}  -f  s  given  below  In  table  2.  Then,  by  use  of  (17),  the 

Table  2.  First  Zero  of  J^z) 


a  z 

a 


-0.5 

1.5708 

0.0 

2.4048 

0.5 

3.1416 

1.0 

3.8317 

1.5 

4.4934 

2.0 

5.1356 

2.5 

5.7635 

3.0 

6.3802 

3.5 

6.9879 

4.0 

7.5883 

4.5 

8.1826 

first  null  location  of  pattern  g(u)  In  (28)  is  at  u0,  where 


The  results  in  figure  14  display  the  first  null  location  as  a  function  of 
8,  for  various  values  of  a.  For  large  B,  u  behaves  as  B  +  iz  /B.  3y  the 

O  c  a 

identification  of  a  as  u  +  v  +  1,  this  curve  applies  to  any  number  of 
dimensions  and  to  whatever  value  of  v  i  s  selected  in  weighting  w(  s)  of  (22). 
The  curves  indicate  that  the  first  null  location  uQ  1S  monotonical ly 
Increasing  in  both  B  and  a. 
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LOCATION  OF  PEAK  OF  FIRST  SIDELOBE 


By  use  of  (28)  and  (21),  we  obtain  derivative 


g/ ( u) 


(40) 


Therefore,  reference  to  (38)  reveals  that  the  location  of  the  peak  of  the 
first  sldelobe  of  g(u)  occurs  where  g'(up)  «  0;  i.e. 


(41) 


If  we  employ  more  explicit  notation  In  (3y)  and  (41),  we  can  express  the  first 
peak  location  In  terms  of  the  first  null  location  according  to 

Up{ 8 ,a)  =  u0(3,a+l).  (42) 

Thus  all  the  results  In  figure  14  can  be  applied  directly  to  the  first  peak 
location.  For  example,  (42)  yields 

i^(B,  -  7)  *  u0(B,  -j);  (43) 

thus  the  third  curve  from  the  bottom  In  figure  14  gives  the  location  of  the 
peak  of  the  first  sldelobe  vrfien  a  =  - 

PEAK  SIDELOBE  LEVEL 


The  value  of  voltage  pattern  g(u)  at  location  (41)  gives  the  level  of  the 
peak  sidelobe: 


*jUZa*l)- 
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Since  the  origin  value  of  the  pattern  Is 

9(0)  “JL(3),  (4b) 

the  voltage  ratio  of  peak  sldelobe  level  to  malnlobe  height  is 


»<V  JA. 0 


(46) 


Relation  (46)  Is  plotted  In  dB  in  figure  15  as  a  function  of  3,  for  various 
values  of  a;  l.e. , 


dS 


(47) 


The  peak  sldelobe  level  decreases  monotonlcal ly  with  increasing  B,  but  has  no 
simple  behavior  versus  a  except  for  very  snail  B  or  very  large  B. 


The  results  of  these  last  two  figures  are  combined  In  figure  16,  where  we 
plot  the  peak  sldelobe  level  In  dB  versus  the  first  null  location  uQ.  These 
latter  curves  are  virtually  linear  over  a  wide  range.  If  ve  di  sregard  the 
sidelobe  decay  rate,  the  most  desirable  region  of  this  figure  is  in  the  lower 
left  quadrant,  l.e.,  9nall  u0  and  very  negative  dB.  However,  the  closest  we 
can  get  (from  our  family)  Is  the  a  *  -.5  curve,  tfiich  1  s,  in  fact,  the  ideal 
pattern;  see  (34)  and  figure  7.  Furthermore,  the  sidelobe  decay  rate  is  tnen 
0  dB/octave.  Higher  sldelobe  decay  rates  are  attained  by  moving  toward  the 
upner  right  quadrant  of  the  figure;  for  example,  the  a  =  3.5  curve  has  a  o«+3 
=  24  dB/octave  sidelobe  decay  rate.  This  figure  furnishes  a  very  compact 
display  of  the  important  interrelationships  that  occur  between  the  fundamental 
features  of  peak  sldelobe  level,  mainlobe  width,  and  sidelobe  decay  rate,  and 
allows  for  a  quick  tradeoff  compari  son  of  alternatives. 
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Figure  15.  Peak  Sidelobe  Level  of  Pattern 


IK  b/bl 
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IDEAL  WEIGHTING  AND  PATTERN 


It  was  shown  In  figure  7,  In  the  previous  section,  that  the  pattern  g(u)  for 

a  =  -  takes  on  a  particularly  desirable  behavior,  namely,  a  narrow  mainlobe 
width  and  a  large  ratio  of  malnlobe-to-peak-  si  delobe.  However,  figures  11  -  13 
Indicated  that  realization  of  some  patterns  was  apparently  not  possible  In 
certain  dimensions  because  we  were  violating  the  condition  on  parameter  v  in 
weighting  (22)  that  allowed  for  convergence  of  Integral  (23).  Here  we  will 
address  the  more  general  problem  of  how  to  realize  pattern  (23), 

9(u)  *JLvl  (V-b2)  for  a11  u»  (48) 

for  any  p  >  -1,  but  without  the  current  restriction  of  v  >  -1  in  weighting 
(22).  This  procedure  will  of  course  require  a  different  and  more  general 
weighting  than  (22),  and  will  f  urnl  sh  solutions  to  tiie  asterisked  cases  in 
figures  11  -  13. 


The  solution  for  the  required  weightings  to  realize  (4d)  for  any  u  >  -1  i  s 
conducted  in  appendix  E.  All  the  weightings  are  zero  for  s  >  1,  as  desired; 
their  values  for  0  i.  s  £  1  are  listed  below.  From  (E-14), 


w(  s) 


$(s-l) 


from  (E-33) 


2vr(v+i) 


6-.r 

2vn  v+i) 


for  v  =  -1 ;  (49) 


for  -2  <  v  <  -1;  (50) 


and  from  ( E -39) , 


w(  s) 


-S'(s-l) 


for  v  =  -2.  (51) 


The  extended  ranqe  for  v  <  -2  i  s  given  in  (E-35)  and  (E-3t>).  Weighting  (49) 
requires  a  generalized  function,  namely,  a  delta  function,  with  its 
singularity  located  at  s  =  1.  Weighting  (51)  requires,  in  addition,  the 
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derivative  of  a  delta  function,  located  at  s  *  1.  The  Intermediate  cases  In 
weighting  (50)  require  a  generalized  function  with  Its  singularity  located  at 
s  *  1;  Interpretation  and  approximation  of  thl s  generalized  function  Is  given 
In  appendix  F. 

It  can  be  observed  from  ( 49)-(  51 )  that  the  leading  term  of  w(  s)  is  exactly 
what  would  have  been  obtained  from  Initial  weighting  (22)  by  substituting  the 
appropriate  value  of  v;  here  ve  are  using  I_n(2)  «  in(2)  ^  9. g.6j. 

However,  the  price  of  crossing  the  “natural  boundary"  at  v  *  -1,  which  was 
originally  required  for  (23),  Is  a  generalized  function  with  its  singularity 
located  at  $  *  1.  And  the  further  we  go  below  v  *  -1,  the  more  singular 
becomes  the  required  generalized  function;  these  points  are  elaborated  upon  in 
appendix  F. 

The  explicit  assignment  of  v  values  In  ( 48)-( 51)  leads  to  table  3  for  the 
weighting -pattern  pairs.  With  regard  to  application  of  (48 ) -( 51 )  to  the  array 

Table  3.  Weighting- Pattern  Pairs;  u  >  -1 

Pattern 

l(i^) 

processing  application  In  various  nimbers  of  dimensions,  we  have  table  4  for 
the  required  weightings,  where  we  have  specialized  tne  values  of  y.  In  all 
cases,  the  pattern  realized  is  the  ideal  one  of  (34): 

gj(u)  *  (7)  cos^u2-B2^  for  all  u.  (52) 


Welghtl ng _ 

(49) 

(50)  with  v  *  - 

(51) 
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Table  4.  Required  Weighting  for  Ideal  Pattern 


Number  of  Dimensions 

Value  of  m 

Required  Weighting 

1 

1 

‘  I 

(49) 

2 

0 

(50)  with  v  »  -  ^ 

3 

1 

1 

(51) 

The  weighting  given  by  (49)  for  one  dimension,  namely,  u  =  -1/2,  has 
already  been  presented  by  van  der  Maas  l'9j.  However,  the  application  of  (49) 
to  the  realization  of  (48)  for  any  u  >  -1  1  s  new.  Additionally,  all  the 
results  In  (50)  and  (51)  for  any  u  >  -1,  and  their  application  to  two-  and 
three-dimensional  array  processing  In  table  4,  are  new.  An  approximation  to 
the  Ideal  pattern  In  two  dimensions,  namely,  weighting  150)  with  v  *  -1.5,  Is 
treated  In  detail  In  appendix  F. 
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SOME  OTHER  WEIGHTINGS 

Another  candidate  class  of  weightings  for  consideration  Is 


of  which  (49)  and  (51)  are  repre sentatlve  examples,  exclusive  of  the 
generalized  functions.  This  class  Is  somewhat  similar  In  form  to  the  earlier 
case  In  (22).  Substitution  of  (53)  In  (15)  yields  pattern 


T hi  s  Integral  converges  at  s  *  0  for  u  >  -1,  but  needs  no  restriction  on  v 
whatsoever. 

To  our  knowledge,  evaluation  of  (54)  Is  not  possible. In  closed  form  for 
general  v;  however,  the  following  cases  are  evaluated  In  appendix  G: 


Numerous  gjeclal  cases  for  one,  two,  or  three  dimensions  are  available  from 
( 55 ) - ( 57 )  by  setting  u  *  -  -j,  0,  or  -j,  re 5>ectively . 
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As  an  example,  If  we  take  {54)  and  (56)  for  v  *  1,  we  have 
1 

y ds  s(u)“'Vusi 

0 

Addition  of  a  £  function  Immediately  yields,  for  u  >  -1, 


which  has  already  been  presented  In  (49)  and  table  3.  A  similar  combination  of 
(54)  and  (57)  yields  the  results  of  (51)  and  the  bottom  entry  of  table  3. 
Additional  results  for  v  =  3,  4,  ...  are  possible  via  the  metnod  of  appendix  6. 

One  other  two-parameter  family  of  weightings  that  affords  a  closed  form 
pattern  Is  available  from  [10;  6.688  1],  by  identifying  u  =1,  x  *  u,  z  =  i8, 
cos  t  *  s,  and  by  using  [5;  9.6.3  and  9.1.35]: 


This  result  Is  restricted  to  the  line  array.  The  weighting  is  continuous  at 
s  *  1  if  v  >  1,  and  the  pattern  (60)  decays  at  3+3 v  dd/octave.  How  good  thi  s 
pattern  Is  has  not  been  pursued. 

All  the  above  results  have  been  aimed  at  getting  closed  form  results  for 
the  pattern;  however,  this  is  by  no  means  necessary.  One  could  consider  the 
class  of  weightings  (53)  for  any  v,  or  the  class 

exp(-32s2)  (l-s2)v  for  0  <  s<  1  (bl) 

for  example,  numerically  by  substitution  in  (12)  or  (15)  and  use  of  some 
integration  rule  like  Simpson's.  Once  the  patterns  have  been  numerical ly 
evaluated  and  plotted  for  a  sufficiently  broad  range  of  values  of  3  and  v, 
good  candidates  can  be  selected  at  will  and  the  corre  spondi  ng  weighting,  ( o  1 ) 
or  (53)  for  example,  easily  evaluated.  For  the  line  array,  this  numerical 

approach  is  readily  accomplished  by  use  of  an  FFT. 
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DISCUSSION 


The  Ideal  pattern  was  defined  In  (34)  as  (2/*/*  cos(jta2-B2),  and  the 
corresponding  weightings  were  given  in  ( 49 ) -( 51 ) .  Now  in  the  one-dimensional 
application,  u  a  -1/2,  van  der  Maas  [9]  has  Indeed  show,  by  taking  the  limit 
of  a  Dolph-Chebyshev  discrete  array  design,  that  there  Is  no  pattern  with  a 
narrower  mainlobe  for  a  specified  sidelobe  level  (and  vice  versa)  than  (34). 
However,  strictly  peaking,  we  have  not  proven  that  same  result  for  the  other 
values  of  u,  l.e. ,  other  numbers  of  dimensions.  Instead,  we  have  adopted  (34) 
as  an  Ideal  pattern  and  shown  that  It  can  be  realized  by  finite-support 
lighting  functions  with  a  generalized  function  whose  singularity  1  s  centered 
at  the  edge  of  the  array.  Conceivably,  there  might  be  a  different  weighting 
that  would  realize  a  pattern  that  gets  further  into  the  left-lower  quadrant  of 
figure  16.  However,  we  conjecture  that  this  Is  not  possible  and  that  the 
leftmost  curve  In  figure  16  is  the  ultimate  attainable  region  for  any 
weighting  In  any  nunber  of  dimensions. 

SUMMARY 

We  have  presented  a  two-parameter  class  of  Bessel  weighting  functions  that 
have  a  closed  form  pattern  with  controllable  mainlobe  width,  mainlobe-to- 
peak-sldelobe  ratio,  and  sidelobe  decay  rate.  These  results  rave  application 
to  arrays  in  one,  tw,  or  three  dimensions.  In  addition,  the  ideal  pattern 
and  the  corre spondl ng  weightings  required  in  various  numbers  of  dimensions 
have  been  derived  and  presented.  Where  a  generalized  function  is  required,  a 
method  of  approximating  it  has  been  presented  and  Illustrated  by  a  numerical 
example.  Various  weightings  already  extant  in  the  literature  were  show  to  ue 
special  cases  or  limiting  cases  of  the  general  results  given  here. 
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Appendix  A 

DERIVATION  OF  RESPONSE  OF  PLANAR  ARRAY 


Let  the  receiving  array  lie  In  the  x-y  plane,  and  let  a  plane  wave  of 

wavelength  x  arrive  at  polar  angle  da  and  azimuthal  angle  ea;  see  figure 
A-l.  Then  If  the  time  of  arrival  of  the  plane  wave  at  the  origin  is  denoted 


z 


X 

Figure  A-l.  Geometry  for  Planar  Array 


as  0,  the  time  of  arrival  at  a  general  point  x,y  in  the  plane  of  the  array  i  s 


where  x  =  c/fQj  c  is  the  speed  of  propagation,  and  f0  is  the  frequency  of 
the  plane-wave  arrival. 


To  look  in  direction  ,  the  receiving  array  should  employ 

steeri  ng-delay 
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at  the  general  point  x,y.  If,  also,  weighting  w2(Xfy)  is  used  the 
receiving  array,  the  array  output  voltage  response  at  frequency  f 

g  »  JJdx  dy  w2(x,y)  exp  (;i2irf0(^a+  ^)] 

*  |Jdx  dy  w2(x,y)  exp £-12*  ^  pi  "  12t  P2^  * 

where  we  define  angle  functions 

^1  *  n<^  cose^  •  si  nj^|  cos^  ,  *  si  slnj  -sinPasinea,  (A-4) 

and  trfiere  the  Integration  Is  carried  out  over  all  x,y  where  weighting  w2  ^  q. 
Thus  the  planar  array  can  have  arbitrary  geometry  In  the  x,y  plane;  equations 
(A-3)  and  (A-4)  are  general  results  for  the  array  response. 

If  weighting  w2  contains  an  Impulse  at  x0,  y0,  then  we  have 


w2(x,y)  *  l  (x-x0)£(y-yQ), 


with  array  response 


g  exp  ^12.  -j  Pj-12.  —  P^]  . 
which  never  decays  In  amplitude  with  Increasing  angle. 
SPECIALIZATION  TO  CIRCULAR  ARRAY 


(A-5) 


(A-6) 


As  a  special  case  of  the  above,  consider  a  pi anar-circular  array  of  radius 
R  with  weighting  independent  of  angle;  i.e.. 


w2(x,y) 


2  ° 

for  x  +yc  < 
otherwi  se 


R  >• 


J 


(A-7) 
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Then  response  (A-3)  becomes 


r  1 

g  *  f  dr  r  J  de  w^r)  exp  ^-12^^  cose  +  P2  s1ne)J 


0  -w 

2,  jj  dr  r  w,{r) 


And  from  (A-4),  we  have 


ah  ?  r  2  2 

\JP i  +P2  *  sin  dj  +  sin  da  -  2s1n^  si cos  (e^-ea) 


1e. 


**  -a 
sln^e  -  s1ndae 


Now  let 


r  =  Rs 


In  (A-8) .  There  follows,  for  re^onse  (A-8), 


i 

g(u)  ■  j  ds  s  J(us)  w(  s) , 


where 


u  *  2*y 


le. 


si  e  ^  -  si  nPae  a 


and 


w(  s)  s  2nR  w^(Rs)  for  0  <  s  <  1. 


(A-8) 


( A — 9 ) 


(A-1J) 


(A-li) 


(A-U) 


(A-1J) 
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Thus  the  voltage  re^onse  g(u)  Is  given  by  a  zero-th  order  bessel 
transform  over  (0,1)  of  normalized  weighting  w(s).  Dimensionless  parameter  u 
Incorporates  the  received  wavelength  x,  the  array  radius  R,  and  the  various 
look  and  arrival  angles. 


Appendix  B 

DERIVATION  OF  RESPONSE  OF  VOLUMETRIC  ARRAY 

Consider  a  plane  wave  of  wavelength  \  arriving  at  angles  ea;  see 
figure  A-l.  The  time  of  arrival  (relative  to  the  origin)  at  a  general  point 
x,y,z  In  the  volumetric  array  Is  [11;  eq.  3J 


T  ■  -  -lx  cose  slnji  +  y  sine  sin  i  +  z  cos«L  (B-l) 

a  c  L  a  a  a  a  aj 

To  look  In  direction  the  delay  used  at  location  x,y,z  should  oe 

*  ijx  cose^  sln^  +  y  sln^  sin  ^  +  z  cosj^"J.  (8-2) 

The  response  of  a  weighted  array,  at  frequency  f  is  then 

g  a  Jj^dx  dy  dz  w3(x,y , z)  exp  [-12irfo(Ta+T^)] 

dx  dy  dz  w3(x,y,z)  exp  £-1^(xP1+yF'2+zPj)]  > 
where  W3  the  weighting  and 


Pj  *  cos  sin  ^  -  cos  efl  sin 

P2  -  sin  sin  ^  -  sin  efl  sin  da, 

P3  =  cos  -  cos  t  .  ( 8-4) 

The  integration  In  (3-3)  is  over  all  x,y,z  where  f.  j.  (8-3)  is  tne 
general  result  for  any  time-delay  steered  volumetric  array. 
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SPECIALIZATION  TO  SPHERICAL  ARRAY 

Let  lighting  W3  depend  only  on  the  di  stance  from  the  array  center;  that 
1  s. 


t  f\)  2  2  2  \  2  2  2  2 

Wj(x,y,z)  *  yk  +y  +z  /  for  x  +y  +z  <  R  , 


(8-5) 


where  R  1  s  the  sphere  radius.  Then  the  voltage  response  of  the  array  Is,  from 
(8-3)  and  (B-5), 


g  *  J* dx  dy  dz  wi^2+y2+z2^  exp  VyP2+zPij] 


R  »  -  _ 

*  J'  dr  r2  J"'  dj)  si ni6  J*  de  w^(r)  ex pjj-1 2lr7^picose  sin^+t^sl  nesi ntf+P^cosrt^j 


R  »  ^ 

2*  ^  dr  r  Wj(r)  ^  dj)  slnfi  exp(‘12rx  P3C0S^  Jo(2^tp7+p7 si  n7 1 


(8-6) 


where  we  have  changed  to  polar  coordinates. 

In  the  Integral  on  r),  let  t  =  cos  4;  the  inner  integral  in  (6-6)  becomes, 
by  use  of  [10;  6.677  6], 
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where  Me  define 


Qs 


( B-8 ) 


Then  the  response  Is,  upon  substitution  of  (B-7)  In  (B-6), 


f  2  Slr(2.£i) 

*4*  dr  rc  w.(r)  - - - . 

J  1  2-t» 


(8-9) 


Now  let  r  *  Rs;  then  the  response  can  be  written  as 


g(u) 


(8-10) 


where 

w(  s)  5  (2w)3/2  R3w1(Rs),  (8-li) 

and  dimensionless  parameter 

u  *  2i£q.  (d- 12) 

The  quantity  Q,  Involving  the  look  and  arrival  angles,  can  be  expressed  as 


Q  *  ^2-2  cos^cosda-2s1ndj  sinda  cos(©e  -©a )j.  (3-13) 

Thus  the  general  result,  (B-10),  for  the  volumetric  array  i  s  given  by  a  sine 
transform  of  the  normalized  weighting  w. 
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Appendix  C 

MONOTONICITY  OF  BESSEL  WEIGHTING 


We  Investigate  here  the  behavior  of  weighting  (22): 

« si  -  f°r  ° 

Let  y  *  3^1- s^;  then  (C-l)  becomes 

w (s)  *  B"2v  yv  Iv(y). 

Now  [5;  9.6.28] 

3y  {y\ly*}  *  yVlv-i(y>  ■ 

which  Is  positive  for  v  >  0,  y  >  0;  see,  for  example,  [5 
1  s  monotonlcally  Increasing  In  y  If  v  i  0;  therefore  (C- 
decreaslng  in  s  if  v  z  0. 


s  <  1.  (C-l) 


(C-2) 


(C-3) 

9.6. 10 J.  Thus  (C-2) 
)  1 s  monotonlcally 
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Appendix  D 

PROGRAM  FOR  CALCULATION  OF  PATTERN  (28) 


10 
20 
30 
40 
50 
60 
70 
SO 
80 
100 
no 
120 
ISO 
140 
150 
160 
170 
ISO 
ISO 
200 
210 
220 
230 
240 
250 
260 
270 
2S0 
280 
300 
3 1 0 
320 


t. 

34  0 
350 
360 
370 

3  SO 
380 
400 
410 

4  20 
4  30 
440 


450 


4  6  0 
47  0 
480 

4  8  O 

5  0  0 
510 


520 


0  1  phi=-,  5 

PLOTTER  IS  "98721:1" 

LIMIT  25, 175,35,245 
OUTPUT  705; "VS5" 

SCALE  O, 6*PI , -70, 0 

GRID  PI, 10 

PENUP 

FOP  6*2  TO  8  STEF'  2 
B2=B*B 

F=FNInuxnu<Al phi, B) 

FOR  U*0  TO  B  STEF'  .  05 
Y=FNInuxnu<Al  phi,  SQR<B2-U*U>  > 

PLOT  U,20*LGT<AB$;<Y/F>  ) 

NEXT  U 

FOR  U«B  TO  6*P I  STEP  .05 
Y=FNJnuxnu<Al  phi, $QR<U*U-B2> > 

PLOT  U,20*LGT<ABS<YxF>> 

NEXT  U 
PENUP 
NEXT  B 
END 

I 

DEF  FNGkiMrn&<  X>  1  GAMMA  C  X  >  Mia  HART,  page  i~r.  4  52  31 
N=INT<X) 

R*X-N 

IF  N  0 ,*  OP  (ROiO  THEN  280 

PRINT  "FHGifMM4<  X;  IS  NOT  DEFINED  FOR 

STOP 

IF  R  O  THEN  320 
Gari'fiii2*l 
GOTO  260 

P*3 .  36954358 1 3 1  +F:*  <  1 . 09S5063045  3  +  P+  .  14292SSC‘94?  +  R*  : .  ?  20  13464  1S6E 
P  =  4  3 . 94 1 0208 1 89  +  F:* < 22 . 9S3OS0O8  36  +  R**..  12.S02 1  686  1  1  2-FoP 
0  =  43. 941O20918l+F'*<4. 39050474596-R>- 7.  150"50622  9--R  •  • 

Garrifiii2  =  P  ■  Q  1  G  A  M  M  A  i  2  ♦  P  ♦'•?**  0  R  1 

IF  N  2  THEN  400 
IF  N< 2  THEN  450 
G  4(11  (H  4=  G  i(u  frt  ii 
GOTO  500 
G  i(ll  (II 4  *  G  4(11  (II  ait 
FOR  1=1  TO  N-2 
G4(ii(ii4=G  k(H(u4*  <  X“K 

NEXT  K 
GOTO  500 
R*  1 

FOR  1=0  TO  1-N 
R  *  R  » ■  X  ♦  1  ) 

NEXT  K 

G  4  hi  Hi  4*G  k(it(iii2  R 
RETURN  ij  kin (ii 4 
FNEHD 
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556  BEF  FN Jnuxnu<Nu,  X>  !  Jnu< x ) /xaivj 

540  IF  ABSTXX1  THEN  650 

550  A*.  797S3456O803 

560  IF  Nu*0  THEN  RETURN  FNJo<X> 

570  IF  Nu*.  5  THEN  RETURN  A*SIN<X>xX 

586  IF  Nu*-.5  THEN  RETURN  A*C0S<X> 

596  IF  Nu»l  THEN  RETURN  FNJIOO/X 

€06  IF  Nu  =  - 1  THEN  RETURN  -FNJ1<X>*X 

610  IF  Nu»1.5  THEN  RETURN  A*<SIN<X>-X*CQSsX/>  X  3 

620  IF  Nu— 1.5  THEN  RETURN  -A*<X*SIN<X>+C0S<X>  > 

630  IF  Nu*2  THEN  RETURN  <2*FNJ1 <X>-X*FNJo- X; >/X'  3 
640  IF  Hu*-2  THEN  RETURN  <2*FNJ1  <X>-X*FNJo<X>  ,'*X 
650  A*Nu 

666  IF  <INT<A>OA>  OR  <A>*0>  THEN  630 
670  K*A“-Nu 

686  S*T*l/<2AA#FNGafnfii*<A+l ) ) 

690  R*-. 25*X*X 

700  Big*ABS<S> 

710  FOP  N* 1  TO  100 

720  T=T*R/<N*<N+A) > 

730  S*S  +  T 

740  Big*MAX<Big,ABS<S) > 

750  IF  ABSCTX-lE-l  1+ABS<S>  THEN  790 
760  NEXT  N 

770  PRINT  "106  TERMS  IN  FNJnuxnuCNu, X >  AT  "JNuJX 
780  PAUSE 

790  D*12-LGT(ABS<B1gS>>  !  NO.  OF  SIGNIF  DIGITS 

860  IF  K  >0  THEN  S*S+<4*RXK 

310  RETURN  S 

820  FNEND 

836  ! 

8-iO  KEF  F N J o < X >  !  Jo<X)  vi  a  9.4.1  &  9.4.3  ot  AMS  55 

S50  Y*ABS  < X  > 

860  IF  V>3  THEN  91 O 

870  T  *Y*Y/9 

880  Jo*.0444479-T*(.0039444-T*. 00021  ) 

390  Jo*  1  -T* < 2. 2499997-T*(  1 . 2656203-T*<  .  31 6  •' 366-T+  Jo  , 

960  GOTO  970 

910  T*3''Y 

920  Jo* 9. 5 1 2E-5-T* <  .  00 1 3 72 37- T 4 < . 00072S05-T- . 000 ;  4476  1 

936  Jo*.  79788456-T*-'?.  7E-7+T*<  .  O055274O+T+ Jo  ,  ■ 

940  S  =  . 00262573-T*' .  00054 125  +  T^T .00029333- '*.0801  ?553 • 
950  T* Y- .  785398 1 6-T *  <  .  04 1 6 6 3 9  7  +  T  +  ''3 . 9  5  4  £  -  5  -  T S  •  :> 

960  Jo*  Jo*C0S<T>  ■'SOP'  Y > 

970  RETURN  Jo 

980  FNEND 

990  1 


1000 
1010 
1020 
10  30 
104  6 
1050 
1060 
1070 
1  0  8  O 
1 0  9  O 
1  1  00 
1110 
1  120 
1  1  30 
1  1  40 
1  1  50 
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DEF  FUJI <X>  1  J1*X)  via  9.4.4  &  9.4.6  st  mm3  35 

Y*ABS < X > 

IF  T>3  THEN  1O70 
T  *  Y  +  Y  9 

J l  =  . O04433 1 9-T  + ( .  00O3 1 76 1 -T» . 0000 l l 09 

J  1  5-T* 1  .  562*' 9985-T# '.  2  1 093573- T  ■»■.  0  :  3542;;.  i  •  . 

GOTO  1 1 36 

T  *  3  •  ■( 

J  1  =  .  0  0  0 1 7 1 0  5  -  T  *  •  .  0024951  1  -  T  +  •  ,  001  i  365  -  T  * .  8  j  0  2  o  o  3  3  1 

J1*.79788456  +  T4,T  .  5 6  E - 6  +  T -  1  .01659667  +  T-J1 
S - .  0 0 6  37879-T+  ■  .  000 74348  + T -  ■  .  0 0 079824-T  - .  O802 °  1  f. 6 


T  *  Y - 2 . 3561 9449  +  T* < .  1 249961 2 +  T-- 5. 65E-5 -T- : 
Jl*SGN*  "►J1+C0S*  T :>  S0R'.'i  > 

RETURN  Jl 
FNEND 


Reproduced  from  gt* 
best  available  copy, 


1160  DEF  FNInuxnu<Nu,  X)  !  lnu<>//'x  nu 

1178  IF  ABS<  XX  1  THEN  1290 
1130  A*. 398942280401 

1190  E*EXP(X> 

1200  IF  Nu-0  .THEN  RETURN  FNIo<X> 

1210  IF  Nu* .  5  THEN  RETURN  A*<E-1/EX'X 
1220  IF  Nu*-. 5  THEN  RETURN  A*<E+1/E> 

1230  IF  Hu* 1  THEN  RETURN  FNI1<X>/X 

1240  IF  Nu  =  - 1  THEN  RETURN  FNI1<X)*X 

1250  IF  Hu-1.5  THEN  RETURN  fl«<(X-l)*E+(X+b  'E)/XA3 

1260  IF  Nu— 1.5  THEN  RETURN  A*< <X-1 > *E- <X+ i >/E> 

1270  IF  Nu*2  THEN  RETURN  <X*FNIo<X>-2*FNI  1  <  XX  3 
1 230  IF  Nu*-2  THEN  RETURN  <X*FNIo<X>-2*FNI  1(X)  >*Y, 

1290  A  *  N  u 

1300  IF  <  INT<  AX  >A)  OR  <A>*0>  THEN  1320 
1310  K*A*-Nu 

1320  S  =  T*l/'<2/'A*FNG-amni*<A+l  >  ) 

1330  R».25*X#X 

1340  Big«ABS<S> 

1350  FOR  N* 1  TO  100 
1360  T*T*R/<N*<N+A>> 

1370  S*S+T 

1330  B i g*MAX< B i g, ABS<8> ) 

1390  IF  ABSCTX-1E-1  H-AB$<S)  THEN  1430 
1400  NEXT  N 

1410  PRINT  "100  TERMS  IN  FN I nuxnu < Nu , X >  AT  "JHuj: 

1420  PAUSE 

1430  D- 1 2-LGTC ABS< B i g/  S> )  !  NO.  OF  SIGMIF.  DIGITS 

1440  IF  K  >0  THEN  S-$*<4*RXK 

1450  RETURN  S 

1460  FHEND 

1470  ! 

1430  DEF  FNIo<X)  !  Io<X)  vi *  9.8. I  t  9.3.2 

1490  Y  *  A  B  S  (  X ) 

1500  IF  V >3 . 75  THEN  1550 
1510  T*Y*Y/14.0625 

1 520  I  o* .  2659732  +  T*< .  O360763+T4 . 00458 1 3  > 

1530  lo*  1  +T*<3. 5156229*T*<3. 0899424 +T*t  1 . 2  u  6  ”  4 2  +  T  -  l  :  •  • 

1540  GOTO  1590 
1550  T-3.75/Y 

1560  I  o*  .  009  1  628 1 -T*  .  02057706-T4 1  .  826355  3 7 -T*  '■  .  0  1  64  76  33- T  *  .  00  ;  92  377  ■  >  • 
1578  Io*.  3989422S  +  TX  .  0 1  328592  +  T*  < .  002253  1  9-T-r  •  .  00 1 5 ~ 5 *  5 -  T ■*  I  >  '■ 

1 530  I  o*  I  o* EXP  <’  Y  > /SQR <  Y  > 

1590  RETURN  Io 
1600  FNEND 
1610  ! 

1620  DEF  F  N I  1  <  X  >  '  1 1  <  X  >  oi  a  9.8.3  9.3.4 

1630  Y  *  A  B  S  <  X ) 

1640  IF  •.>3.75  THEN  1690 
1650  T*Y4  fV  i 4 . 0625 

1660  1 1*  .  02658733  +  T**' .00301532  +  T4. 0003241 1  1 

1670  I  1 « ;  ■!  *  <  .  5  +  T  *  < . .  8  7  8  9  O  5  9  4  +  T  *  i  .  5  1  4  9  8  8  6  9  T  *  •  .  l !;  ;  4 -  -  -  I  1  • 

1680  GOTO  1730 
1690  T  *  3 .  7  5  x  Y 

1700  I  1  =  .  0  1  03  1  555  -T4r. .  02232967-T*  .  82395  3  1  2  -7-* 1  .  "  :  7 -  5  -  T  *  .  00 4 2005  3 
1710  I  1  =  .  39894228-T*'  .  03988024+T  *  >  .  003620  1  -T*  .  02  1  .  :  0  1  -  *11  1 
1720  I  1  =  3GN  •  ;  •  I  1 -*EXP  •  Y '•  S Q F:  Y  > 

1730  RETURN  II 
1740  FNEND 
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Appendix  E 

DERIVATION  OF  WEIGHTING  FOR  IDEAL  PATTERN 
We  want  to  realize  pattern  (23), 

9(u)  for  aH  u»  (l 

for  any  u  >  -1,  but  without  the  restriction  v  >  -1,  which  was  required  for 
convergence  of  Integral  (22).  ^a(z)  was  defined  In  (17)  et  seq. 

We  begin  the  derivation  for  the  required  weighting  to  realize  (E-l)  by 
substituting  (E-l)  In  (16): 


w(  s) 


for  s  >  0. 


( E-2) 


It  Is  Important  to  observe  that  we  must  allow  all  s  >  0  In  (E-2);  hopefully, 
vrfien  we  evaluate  w(  s)  from  (E-2),  It  will  be  zero  for  s  >  1. 


Now  we  already  know  from  (22)  and  (23)  that  (E-2)  yields 


s)  * 


i f  u  >  -1 ,  v  >  -1 .  (E-3 ) 


Letting  a  *  m-v-vl  In  (E-?)  and  (E-3),  and  eliminating  v,  we  have  the  useful 
Integral  identity 


For  convergence  of  this  integral  at  u  =  0,  we  require  u  >  -1,  whereas  for 
convergence  at  u  =<*>,  we  must  have  a  >  y;  i.e. ,  -1  <  u  <  a. 
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Now  ws  have  the  relation  [5;  9.1.30] 


u(s)“  J„<  su)  •  -  Igl  ju(s)“  ‘j.j!  sulj  for  s  >  0. 
Thus  (E-2)  can  be  expressed  as 


Appeal  to  (E-4)  now  reveals  that 


provided  that  u  >  0,  v  >  -2. 


(E-7) 


We  have  succeeded  In  extending  the  range  of  v  from  v  >  -1  to  v  >  -2,  as 
desired,  but  have  apparently  altered  and  restricted  the  range  of  u  from  u  >  -1 
to  u  >0.  However,  this  last  restriction  Is  due  solely  to  the  method  of 
derivation,  and  may  be  restored  to  u  >  -1,  by  observing  that  the  right-hand 
side  of  (E-7)  is  analytic  In  u  (In  fact,  constant),  and  that  the  function  w(  s) 
defined  by  (E-2)  Is  analytic*  In  u  for  u  >  -1.  Thus  (E-7)  gives  the  required 
weighting  to  realize  pattern  (E-l),  provided  that 


ii  >  -1,  v  >  -2. 


(E-8) 


However,  care  must  be  taken,  In  the  evaluation  of  the  derivative  in  (t-7),  to 
account  for  any  generalized  functions  that  may  be  generated. 


*  We  are  using  the  fact  that  $a(z)  is  an  entire  function  of  a,  regardless 
of  the  value  of  z;  see  (17)  et  seq. 
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In  (E-7),  define  the  function 


Then  there  follows 

M  s)  *  -  i  Wj{  s)  for  all  s,  if  u  >  -1,  v  >  -2.  (E-IO) 

We  now  break  the  region  v  >  -2  Into  the  three  subcases  (1)  v  >  -1, 

(11)  v  »  -1,  (111)  -2  <  v  <  -1. 

(1)  v  >  -1 


As  s-*  1- ,  Wj(  s)  in  ( £-9 )  approaches  0,  since  v+l  >  0.  Tnus  wjJ  s)  in 
(E-9)  Is  continuous  for  all  s,  and  we  find,  by  the  use  of  [5;  9.6.28J  in 
(E-7),  that 


which  checks  (22),  as  it  must  of  course,  for  v  >  -1. 
We  observe  that 


^  s)  p [ vVl)  as  (t-12) 

Thus  for  -1  <  v  <  0,  there  1  s  an  integrable  singularity  in  w(s)  at  s  =  1.  For 
v  >  0,  w(  s)  Is  continuous  at  s  =  1,  while  for  v  =  0,  w(  s)  has  a  di  scontinuous 
step  of  value  -1  at  s  =  1. 
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(11)  v  -  -1 


Now,  from  (E-9), 


(£-13) 


This  function  has  a  discontinuous  step  of  value  -1  at  s  *  1.  Thus,  from 
(E-10)  and  [5;  9.6.28  and  9.6.6], 


The  Bessel  function  portion  of  this  weighting  approaches  the  finite  value 
B2/2  as  s-*l-.  The  integrate  singularity  at  s  *  1  of  the  previous  subcase 
for  v  >  -1  has  evolved  here  into  a  S -function  for  v  *  -1  at  s  ■  1. 


This  weighting  w(  s)  in  (E-14)  realizes  the  desired  pattern  in  (E-l)  for 
v  *  -1,  namely. 


>  -i- 


(£-!.*>) 


The  special  case  of  a  linear  array,  u  =  -1/2,  yields  the  ideal  pattern 


MM  4T^M- 

This  weighting-pattern  pair,  (E-14)  and  (E-16),  is  already  known  for  tne  line 
array  under  the  name  of  van  der  Maas  [9], 
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An  alternative  way  of  obtaining  the  result  ( E-14 )  Is  by  taking  the  limit 
of  (£-11)  as  v-w-1.  First  we  observe  that 


for  s  <  1,  (E-17) 


since  I v( z)  Is  an  entire  function  of  \>  wnen  z  t  0  C5;9.6.1J.  We  then 
define  a  difference  or  remainder  function  (for  v  >  -1)  as 


R 


:i(aR) 


for  s  <  1,  (£-18) 


where  we  used  [5;  9.6.6].  The  area  under  the  remainder  function  s  Rv(  $)  in 
a  small  region  near  s  *  1  Is 

A 

1  ( 2e-£) 

J  ds  s  Rv(  s)  * 


J  dt^|)VIv(Bt)  -  B  I^Btjj 


1  for  v  >  -1,  (E-19) 


where  we  used  [5;  9.6.28].  Therefore, 


1 1m 

V-*  -1 


1* 


ds  s  Rv(  s)  =  1,  regardless  of  e  (>0). 


(t-20) 


Thus  since  factor  s  is  1  at  the  upper  limit  of  integration, 


1  im 

v-»  -i  V 


Rv(  s)  =  $(s-l). 


(£-21) 
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Therefore,  combining  (£-17)  and  (£-21),  we  obtain 


1  I,(b|£?)  +  £( s_1)  for 

By  v\  '  /  -y — ^  1\  1  / 


s  1  1,  ( E -22) 


In  agreement  with  (E-14) 


An  alternative  and  simpler  equivalent  form  of  (E-22)  Is 


"-1  (V?)\(V?)  * ♦  2lts) 


for  s  >  0.  ( £ -23) 

The  derivation  of  (E-23)  Is  similar  to  that  given  above  In  (E-13)-(£-21) . 


(Ill)  -2  <  v  <  -1 


We  return  to  (£-7)  and  (E-9).  Observe  that  [5;  9.6.7J 


W,(s) - 


V  +  l 


2v+1nv+2) 


as  s-*l-. 


(£-24) 


SI  nee  we  now  have 


-1  <  v+l  <  0, 


( E-25) 


there  Is  an  Integrable  singularity  In  wi(s)  at  s  =  1.  Thus  the  derivative 
In  (E-10)  will  generate  a  generalized  function  with  a  singularity  located  at 
s  =  1.  We  handle  this  case  by  defining  an  auxiliary  function 


A(  s)  h 


for  0  < 

for  1  < 


s  <  1, 


s 


j 


(£-2o) 
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Then  using  (£-9),  the  difference 


Is  a  continuous  function  of  s;  In  fact, 

2/  2)v+ 2 

w,(s)  -  A(s)~  B  -  as  s-*l-,  (E-28) 

1  2V+3T( v+3) 

which  approaches  zero  since  v  >  -2.  Therefore,  the  required  weighting  w(  s)  In 
(£-10)  can  be  expressed  as 


w(  s) 


1  _d 
s  ds 


[w‘ 


(  s)  -  A(  s)  +  A(  s) 


•} 


2T(v+i) 

J 


s  D(s)  +  G(  s) , 


(E-29) 


where  both  D(  s)  and  G(  s)  are  zero  for  1  <  s.  The  difference  function  D(  s) 
possesses  an  integrable  singularity  at  s  *  1;  in  fact  (recall  ( E -25) ) , 


B26-s2)Vtl 

v*2r< 


as  s-*l-. 


(E -30 ) 


The  last  term  in  w(  s)  in  (£-29)  is  a  generalized  function;  from  (E-2t>), 
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We  have  now  completed  the  consideration  of  the  three  subcases  delineated 
under  (E-10).  We  now  wish  to  extend  v  to  values  that  are  equal  to  and  less 
than  -2,  so  that  we  can  handle  the  volumetric  array  discussed  in  (37)  et  seq. 


We  return  to  (E-6)  and  employ  (£-5)  again: 
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(c-Jh) 


Recourse  to  (E-4)  yields 


provided  that  v  >  -3,  u  >  1.  However,  the  last  restriction  on  u  may  be 
restored  to  u  >  -1,  by  the  argument  under  (E-7).  Thus  (E-35)  gives  the 
required  weighting  to  realize  pattern  (E-l),  provided  that 


u  >  -1,  v  >  -3. 


U-Jo) 


The  only  subcase  of  ( E -35)  that  we  consider  In  detail  Is: 
( 1 v)  v  *  -2 

We  now  can  write  (E-35)  as 


Therefore  [5;  9.6.28], 


(E-37) 


(E-38) 
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Here  Me  have  used  (21)  and 

^'(s-1)  -  J(s-l)  +  J'(s-l), 


vrfiich  follows  from 

f(x) J'(x-a)  -  [f( a)  +  f ' ( a) ( x-a)  +  of(x-a)2]]  $'(x-a) 

*  f( a) (x-a)  -  f'(a)£(x-a). 

This  required  welghtl  ng,  (E-39)  for  v  *  -2,has  both  a  $ function  and  a  S 
function  at  s  =  1. 

Summary 

The  required  weightings  to  realize  pattern  (E-l)  are  given  by 


r« 

-11) 

for 

V  >  -1 

),e 

-14) 

for 

v  =  -1 

(E 

-33) 

for 

-2  <  v 

(E 

-39) 

for 

v  *  -2 

LU 

-35) 

for 

-3  <  v 

V 


(E-3*) 


(E-40) 


( E-41) 


for  n  >  -1. 


(E-4^) 


Appendix  F 

APPROXIMATION  OF  A  GENERALIZED  FUNCTION 

Equation  (50)  or  ( E -33)  gives  the  required  weighting,  for  -2  <  v  <  -1,  to 
realize  pattern  (48)  In  terms  of  a  generalized  function  which  is  difficult  to 
Interpret.  Here  we  address  thl  s  intrepretation  by  means  of  an  approximation 
to  the  generalized  function  (E-31).  We  begin  by  approximating  the  (singular) 
auxiliary  function  A(s)  defined  in  (£-26)  by  an  ordinary  function  A£.(s)>  and 
then  derive  an  approximation  to  generalized  function  G(  s)  of  (E-31)  according 
to  the  same  rule,  namely. 


Ge(5)  "  '  1  d7\(s)- 

In  particular,  consider  A£(sj  as  Shown  in  figure  F-l;  that  is,  Ae(  s) 
is  still  given  by  ( E - 26 )  for  0  i  s  £  1-e,  but  then  tapers  linearly  to  zero  at 
s  =  1  in  order  to  be  everywhere  continuous.  The  height  d  of  A£(  s)  at  s  =  1-e 
i  s,  from  ( E -26) , 


H 


.  e v+1( 1-e/ 2) V+1 

rr^zi — 


(>  0). 


(F-2) 


For  snail  c,  we  have 


v+1 

H  ~  fTv+ZT  as  (F“3) 

which  tends  to  +«>  as  e-*0+,  since  we  have,  from  (£-25), 

-1  <  v+1  <  0.  (F-4) 

The  result  of  applying  (F-l)  to  figure  F-l  is  shown  in  figure  F-2.  The 
large  positive  pulse  in  (1-e,  1)  has  height  proportional  to 


H  .  eV(l-e/2)V+1 
e  T  (v+2) 


fTwTT 


a  S  e -*0+ , 


(F-5) 
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Figure  F-l.  Approximation  to  Auxiliary  Function  A(s) 


Figure  F-2.  Approximation  to  Generalized  Function  G(s) 


which  Is  tending  to  Infinity  since  -2  <  v  <  -1.  The  area  of  this  positive 
pul se  Is 


C 


f 

—  r 


-  ln(l-e)~ 


eV+1 

fT^2l 


as  e-»0+. 


(F-6J 


This  area  Is  also  tending  to  +*  as  e-wO+;  recall  (F-4).  Tne  rate  of  Increase 

of  C  1  s  greater,  the  closer  v  Is  to  -2.  (For  v-*  -1,  H-+1,  C-*  ln(l-c);  thus 
area  Ol  as  c-*0+.  This  Is  the  unit  area  Impulse  presented  In  (49)  for 
v  ■  -1.) 


Figure  F-2  Is  one  approximation  to  generalized  function  b(  s)  defined  In 
(E-31).  Its  most  Important  feature  Is  the  Impulsive-like  positive  pulse  near 
s  *  1.  An  alternative  approximation  Is  afforded  In  figure  F-3,  where  the  area 
C  of  the  Impulse  at  s  *  1  Is  given  by  (recall  (F-6)) 

-  cv+1 

c  'rr«zr  <'"7, 

The  notation  used  In  (E-31)  for  the  generalized  function. 


G(  s) 


2vr(v+u 


(F-B) 


conceals  the  positive  Impulsive  behavior  at  s  *  1  that  the  series  of 
approximations  In  figures  F-l  through  F-3  indicate  must  be  present.  In  fact, 
(F-8)  Is  negative  for  0  £  s  <  1,  by  reference  to  (F-4). 


The  alternative  approximation  we  obtain  to  weighting  (50)  is  then,  from 
( E - 29 )  and  figure  F-3, 
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Figure  F-3.  An  Alternative  Approximation  to  Generalized  Function  G(s) 
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I  l\  u/  Wi 


wtf(  s)  ■  0(  s)  ♦  6£(  s) 


The  plot  In  figure  F-4  Illustrates  this  approximation.  The  “transition 
portion"  In  (1-c,  1),  vrtilch  Is  the  bottom  line  of  (F-9),  Is  singular  at  s  *  1-; 
however,  this  Is  an  Integrable  singularity,  as  may  be  seen  by  reference  to 
(E-29)  and  (E-30).  The  Impulse  at  s  *  1  1  s  of  finite  area  C  given  by  (F-7). 

As  e-+0+,  area  C  of  the  Impulse  tends  to  Infinity;  see  (F-7)  and  (F-4). 
However,  the  area  under  the  main  portion  of  the  approximation  w£(s) 
precl  sely  cancels  this  singular  behavior;  that  Is,  as  e-*0+, 


ds 


(l-s)v 

r(v+-i) 


(F-iu) 


which  is  -C.  Since  the  area  under  the  transition  portion  is 


$  ds0(s)~  I  ds  Irfcll  ~ 

1-e  1 - e 


B2ev+2 

7rray 


as  E-wO+, 


( F-ll) 


which  tends  to  0  as  e-»0+,  the  area  under  approximation  w£(s)  remains  f i nite 
as  e-*0+.  Indeed  it  must  remain  finite,  because  Hankel  transform  (12)  or  lib) 
must  rema*.'  finite  in  order  to  realize  pattern  (48),  which  is  entire  in  u ,  v, 
u,  and  8. 
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Thus  our  final  approximation  to  w(  s)  1  s  simply 


w  (s)  « 


for  0  <  S  <  1-e 


for  1-e  <  s  <  1 


+C<f(s-1),  (f-12) 


J 


and  Is  shown  In  figure  F-5.  It  Is  very  important  to  observe  that  the  simple 
expedient  of  approximating  w(  s)  by 


(f-13) 


Is  totally  inadequate  because,  as  e-*0+,  the  area  under  tne  cusp  at  s  5  1-e 
tends  to  Infinity,  and  cannot  possibly  yield  an  entire  function  for  the 
pattern.  The  impulse  1  s  necessary  to  compensate  for  the  singular  behavior  of 
(F -13)  near  s  *  1;  It  allows  us  to  realize  the  "finite  part"  of  the  Hankel 
transform  of  (F-13)  for  c  *  0. 


(As  v-*-l,  the  value  of  C  In  (F-7)  tends  to  the  finite  value  1,  which  is 
the  Impulse  In  (49).  And  as  v-»-2,  the  doublet  of  (51)  could  probably  be 
extracted  as  a  limit  from  (F-12);  this  procedure  nas  not  been  pursued.) 


The  result  of  using  approximation  (F-12)  with  (F-7),  for  v  =  -1.5  and 
B  «  4,  Is  displayed  as  the  patterns  In  figures  F-6  through  F-d  for  e  =  .1, 

.01,  .001,  respectively.  We  have  selected  u  -  0,  that  is,  two  dimensions,  and 
are  approximating  the  ideal  pattern  for  B  =  4  shown  in  figure  Id.  It  is  seen 
that  the  approximations  become  progressively  better  as  e  decreases,  and  that 
the  result  In  figure  F-8  is  indeed  very  close  tr  figure  13. 

The  approximation  w£(  s)  In  (F-12)  and  figure  F-5  used,  for  the  impulse 
area,  the  value  C  given  by  (F-7)  as  a  limit  of  (F-6)  for  snail  e.  A  better 
approach  is  not  to  use  the  limiting  value,  but  to  use  the  actual  value  of  the 
pertinent  function,  since  we  would  like  good  approximations  for  moderate 
values  of  e,  not  just  very  snail  e.  This  procedure  is  considered  in  detail  in 
[12],  with  the  result 
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The  patterns  for  this  approximation  are  depicted  In  figures  F-9  through  F-ll. 
The  result  In  figure  F-ll  for  e  s  .1  Is  now  better  than  the  result  for 
c  *  .001  In  figure  F-8;  and  all  we  have  done  1 s  to  modify  the  area  of  the 
Impulse  at  s  *  1.  The  result  for  e  *  .15  in  figure  F-10  Indicates  a  modest 
change  from  the  Ideal  and  would  be  acceptable  In  some  cases.  Tne  program  for 
the  pattern  evaluation  Is  listed  at  the  end  of  this  appendix. 

Another  possibility  Is  to  relocate  the  Impulse  In  (1-e,  1)  to  best 
approximate  the  Ideal  pattern  In  figure  13.  More  generally,  a  shaped  narrow 
oulse,  which  Is  concentrated  toward  the  boundary  at  s  *  1,  could  be  used; 
these  possibilities  are  discussed  further  In  [12]. 

Some  additional  results  Involving  delta  functions  and  Vessel  transforms 
are  given  here  In  appendix  H. 


Figure  F-5.  Final  Approximation  to  Weighting  (50) 
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0  n 

Figure  F-i 


2n  3  n  t 

u 

Pattern  of  Approximation  (F-12 
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IK  0/0.1 


Figure  F-9.  Pattern  of  Approximation  (F-14)  for  z  =  .2 
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Figure  F-ll.  Pattern  of  Approximation  ( F- 14 )  for  e  =  .1 
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PROGRAM  FOR  PATTERN  EVALUATION 

IP  tps=.l  !  Pattern  for  Weighting  <.F-*4> 

20  Mu*0 

3©  Nu=-1.5 

40  Be  *4 

50  DIM  G<0:240> 

60  COM  U,  Be , Mu, Nu, M21 

70  M2 1 *Mu*2+ 1 

30  T=2*Eps-Eps'-2 

30  F le«T''(Nu+l  >  *FN  I nuxnut  Nu+ 1 ,  Ec*SQR  ( T  >  ;■ 

100  fi=0 

110  B  = 1 - E  p i 

120  FOR  I u*0  TO  240 

130  U*Iu*PI/40 

140  S»<FNS<A)*FN8<B>i*.5 

150  N  =  2 

160  H=  <  B-fl  >  * . 5 

170  F=<B-fi)''3 

130  Vo=9E99 

190  T  =  0 

200  FOR  k* 1  TO  M-l  STEP  2 
210  T«T+FNS'fl+H*IO 

220  NEXT  K 

230  $*S  +  T 

240  V“<S*T)*F 

250  IF  rlBS  i  V-Vo  >  <  =  iiB8  <  V  )  *  1 E-4  THEN  310 

2&0  Vo*V 

270  N  =  N*2 

280  H*H* . 5 

296  F=F*.5 

300  GOTO  190 

3  1  O  G<  lu ) *F  1  e*FH Jnuxnu •, Mu ,  U  ) ♦V  1  Vo  1 1  age  Rea  p on 

320  PRINT  I  u ,  G  < I  u  > 

330  NEXT  lu 

340  PLOTTER  IS  B9372Fr 

350  LIMIT  25,175,35,245 
360  OUTPUT  705; “VS5" 

370  SCftLE  0,240,-70,0 

380  GRID  40,10 

390  PENUP 

400  FOP  Iu-O  TO  240 

410  PLOT  I  u ,  20*LCT  •,  FIBS  f  G  <  I  u  >  /  G  <  0  >  >  > 

420  NEXT  lu 

430  PENUP 

440  END 

450  ! 

4  60  DEF  FNS 1 8  > 

470  COM  U, Be , Mu, Nu, M21 

480  T  =  1  -  8  *  3 

490  T  1  =  F  N  J  n  u  <  n  u  <  M  u ,  U  »•  8  > 

50O  T2  =  FNInuxnu'  Nu, Be *SQR(T >  > 

510  RETURN  8  M21*T  Nu*Tl*T2 
520  FNEND 

510  1 


FN.f»iu  nu  and  FmI-’u  ru  -ire  1  l  =  t  £  2  :n  Hf:pe'i3i 
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Appendix  G 

EVALUATION  OF  A  BESSEL  INTEGRAL  VIA  RECURSION 
The  Integral  of  Interest  Is 


where  the  kernel  K  Is  the  Bessel  function  as  given  in  (11).  We  have,  via 
[5;  9.6.28], 

1 

^  g^(u, B)  *  ^  ds  K(u,s) 

0 

*  3  gv_1(u,J).  (C-2) 

(This  relation  actually  holds  for  any  kernel  K,  not  just  (11).)  Since,  from 
(G-l) , 

9v(u,0)  *  0  if  v  >  0, 
we  have  the  Integral  recursion 


B 

g^fu.B)  *  \  dt  t  gvl(u,t)  for  v  >  0.  (G-3) 

0 

We  already  know  the  starting  case  of 

1 

g0(u,B)  «  ^  ds  J  (us)  IQ 
see  (23).  Substitution  In  (G-3)  yields 
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g  u 

9l(u,8)  ■  j  dt  J 


yn^B 


where  we  employed  (21). 


Now  we  employ  (G-3)  and  (G-5): 


g2(u,B)  •  J  dt  t 


the  Integral  evaluation  follows  from  direct  comparison  with  (G-5) 


The  last  case  for  v  *  3  follows  In  similar  fashion: 


93<u*B>  =/,-2(r-B7  Jv2(u)  -■2b2/,-1(u)  *yjl(u) 


(G-5) 


(G-6) 


(G-7 ) 


76 


TR  6/61 


Appendix  H 

TWO  BESSEL  INTEGRALS  THAT  YIELD  GENERALIZED  FUNCTIONS 


The  starting  point  Is  the  Hankel  transform  pair  in  (15)  and  (lb) 
let  w(  s)  ■  J(s-a)  (where  a  >  o)  In  (15),  we  get 


g(u)  -  a  f-J  J  (au)  for  u  >  0  . 


Substituting  (H -1 )  In  (16)  then  yields  the  useful  relation 


(  du  u  J  (  su)  J  (au)  *  7  £{  s-a) . 

J  W  U  a 


On  the  other  hand.  If  we  let  the  weighting  be  a  doublet. 


w(s)  *-^#(s-a),  then  9(u)  =  -u(uJ Jw-l(au)' 


The  Inverse  relation  (16)  yields 


oo 

-(f)  \  J,(s-a)  "I du  u‘ 


J  ( su)  J  , (au) . 
u  u-l 


However,  since 


f(  s)  J'(  s-a)  =  [f(a)  +  f’(a)(s-a)  ♦  ...]j'(s-a) 
*  f(  a)  J'  ( s-a)  -  f '  (  a)£(  s-a) , 


then 


O® 

J  du  u2  su)  J  .jfau)  =  -  ^J'(  s-a)  +  ^  l(  s-a) 


Equations  (H-2)  and  (H-6)  are  the  desired  results. 


If  we 


( H-l ) 


(H-2) 


(H-3) 


(h-4) 


(H-5) 


(H-6) 


77/78 
Reverse  Blank 


TR  6761 


REFERENCES 

1.  F.  J.  Harris,  "On  the  Use  of  Windows  for  Harmonic  Analysis  with  the 

Discrete  Fourier  Transform,"  Proc.  IEEE,  vol.  66,  no.  1,  January  197b,  pp. 
51-83. 


2.  A.  H.  Nuttall,  "Some  Windows  with  Very  Good  Sidelobe  Behavior,"  IEEE 
Trans,  on  Acoustics,  Speech,  and  Signal  Processing,  vol.  ASSP-29,  no.  1, 
February  1981,  op.  84-91;  also  NUSC  Technical  Report  6239,  9  April  1980. 

3.  J.  F.  Kaiser,  "Digital  Filters"  In  System  Analysis  by  Digital  Computer, 

F.  F.  Kuo  and  J.  F.  Kaiser,  editors,  J.  Wiley  and  Sons,  NY,  1966,  pp. 
218-285. 

4.  J.  F.  Kaiser  and  R.  W.  Schafer,  “On  the  Use  of  the  IQ-S1nh  Window  for 
Spectrum  Analysl  s,"  IEEE  Trans,  on  Acoustics,  Speech,  and  Signal 
Processing ,  vol.  ASSP-28,  no.  1,  February  1980,  pp.  iU5-lo7. 

5.  Handbook  of  Mathematical  Functions,  U.  S.  Dept,  of  Commerce,  National 
Bureau  of  Standards,  Applied  Math.  Series  55,  U.S.  Govt.  Printing  office, 
June  1964. 

6.  W.  Magnus  and  F.  Oberhettlnger,  Formulas  and  Theorems  for  the  Functions  of 
Mathematical  Physics,  Chelsea  Publishing  Co.,  N.Y.,  1954. 

7.  G.  N.  Watson,  Theory  of  Bessel  Functions,  Cambridge  University  Press, 
London,  England,  Second  Edition,  1958. 

8.  R.  L.  Streit,  "A  Two- Parameter  Family  of  Weignts  for  Nonrecursive  Digital 
Filters  and  Antennas,"  submitted  to  IEEE  Trans,  on  Antennas,  21  April  1982. 

9.  G.  J.  van  der  Maas,  "A  Simplified  Calculation  for  Dolph-Cheby  shev  arrays," 
Journal  of  Applied  Physics,  vol.  25,  no.  1,  January  1954,  pp.  121-124. 

10.  I.  S.  Gradshteyn  and  I.  M.  Ryzhik,  Table  oi  Integrals,  Series,  and 
Products,  Academic  Press,  Inc.,  N.Y.,  1980. 


79 


TR  6761 


11.  A.  H.  Nuttall,  Probability  Distribution  of  Array  Response  for  Randomly 
Perturbed  E lenient  Gains,  NUSC  Technical  Report  5687,  29  September  1977. 

12.  A.  H.  Nuttall,  Approximations  to  Some  Generalized  Functions;  Application 
to  Array  Processing,  NUSC  Technical  Report  6767,  19  July  1982 


80 


Technical  Report  6767 
19  July  1982 


Approximations  to  Some 
Generalized  Functions; 
Application  to  Array 
Processing 


A.  H.  Nuttatl 

ABSTRACT 

Approximations  to  some  generalized  functions  generated  by  the 
process  of  differentiation  are  developed  and  used  to  interpret  some 
integrals  involving  generalized  functions.  Application  to  the 
realization  of  the  ideal  voltage  response  pattern  of  an  array  is  made 
and  illustrated  for  both  an  impulse  and  a  smooth  approximation  to 
the  required  generalized  weighting  function. 


Apprcved  (or  public  release;  distribution  unlimited 


TR  6767 


TABLE  OF  CONTENTS 

Page 


LIST  OF  ILLUSTRATIONS  11 

LIST  OF  SYMBOLS  111 

INTRODUCTION  1 

APPROXIMATION  PROCEDURE  2 

EXAMPLES  8 

APPLICATION  TO  ARRAY  WEIGHTING  14 

USE  OF  NEUTRALIZERS  19 

SUMMARY  29 

APPENDIX  A  -  VOLTAGE  RESPONSE  PATTERN  FOR  WEIGHTING  (36)  31 

APPENDIX  B  -  PROGRAM  FOR  THE  EVALUATION  OF  (59)  37 

REFERENCES  41 


1 


TR  6767 


LIST  OF  ILLUSTRATIONS 

Figure  Page 


1  Approximation  f£(x)  to  f(x)  3 

2  Approximation  ge(x)  to  g(x)  3 

3  Approximation  to  Generalized  Function  xw  U(x)  8 

4  Approximation  to  Generalized  Function  1/x^  il 

5  Pattern  of  Spherical  Array  for  Weighting  (36)  17 

6  Pattern  of  Circular  Array  for  Weighting  (36)  18 

7  Neutralizer  n£(x)  and  its  Derivative  19 

8  Approximation-  to  Generalized  Function  via  Neutralizer  21 

9  Neutralizer  (55)  and  its  Derivative  24 

10  Approximate  Weighting  wc(x)  f0r  v  =  -1.5,  B  «  4  25 

11  Pattern  of  Spherical  Array  for  Weighting  (53)  27 

12  Pattern  of  Circular  Array  for  Weighting  (53)  28 


TR  6767 


LIST  OF  SYMBOLS 


f(x)  ordinary  function 

g(x)  generalized  function 

h(x)  good  function 

c  parameter  controlling  the  transition  region 

fc(x)  approximation  to  f(x) 

gc(x)  approximation  to  g(x) 

U ( x )  unit  step  function 

£(x)  delta  function 

I  approximation  to  integral  I 

v,B  weighting  parameters;  (32) 

w  dimensionality  parameter  of  array;  (38) 

w(x)  required  array  weighting;  (32),  (37) 
we(x)  approximate  array  weighting;  (36),  (53) 

u  array  parameter  containing  geometry,  wavelength,  and  look  and 

steering  angles;  (38) 
v(u)  array  voltage  response;  (38) 

^a(z)  Bessel  function  ratio;  (40) 

vi(u)  ideal  voltage  pattern;  (41) 

vc(u)  approximate  voltage  pattern;  (59) 

nc(x)  neutralizer;  figure  7 

y  linear  transformation  of  x;  (54) 


i  i  i  /  iv 
Reverse  Blank 


TR  6767 


INTRODUCTION 

Evaluation  of  integrals  involving  generalized  functions  is  often 
accomplished  via  an  integration  by  parts,  without  regard  to  an  interpretation 
of  the  behavior  of  the  function  at  its  singular  points.  See,  for  example, 
ref.  1,  eqs.  (1-17),  (1-18),  and  (1-32);  thus,  there  follows  an  integral  such 
as 


oo 

f  dx  .  (1) 

—  «d 

It  is  difficult  to  interpret  and  attach  physical  significance  to  this 
integral;  in  fact,  the  major  contribution  to  the  integral  in  (1)  comes  from 
the  neighborhood  of  x  -  0,  where  the  integrand  appears  to  be  positive  and  not 
integrable,  yet  the  right-hand  side  of  (1)  is  negative  and  finite.  We  would 
like  to  approximate  the  generalized  function  1/x2  in  (1)  and  get  a  physical 
interpretation  that  is  consistent  with  the  result  given  by  (1). 

In  a  recent  study  of  the  ideal  patterns  for  arrays  in  one,  two,  and  three 
dimensions,  it  was  found  that  the  required  weightings  were  impulsive  or  more 
singular  than  an  impulse,  depending  on  the  dimensionality  of  the  application; 
see  ref.  2.  In  order  to  make  these  results  of  practical  utility,  it  is 
necessary  to  approximate  these  singular  weightings  (generalized  functions)  by 
finite-valued  functions  and  thereby  realize  approximations  to  the  ideal 
patterns.  This  approximation  procedure  and  its  performance  will  be  detailed 
here. 
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APPROXIMATION  PROCEDURE 


We  take  as  given  the  possibility  of  approximating  a  delta  function  £(x) 
by  finite-valued  functions;  see,  for  example,  ref.  1,  page  279,  where  a 
one-sided  rectangular  pulse  and  a  two-sided  Gaussian  pulse  are  used  for 
illustration  purposes.  Also,  on  page  280,  a  one-sided  approximation  to  the 
doublet  S'(x)  is  given.  Alternative  approximations  are  presented  in  ref.  3, 
pages  11-12. 


Let  us  suppose  that  f(x)  is  an  ordinary  function  which  is  integrable  at 
x  -  a+,  but  that  generalized  function  (ref.  3,  page  30) 


g(x)  s  f'(x) 


(2) 


is  not  integrable  at  x  *  a+.  For  example,  with  a  =  0, 


f(x) 


g(x) 


|-2x-'^  for  x  >  0 


for  x  <  0 


for  x  >  0 
for  x  <  0 


(3) 


(4) 


is  such  a  pair;  see  ref.  3,  definition  8.  Yet  integration  by  parts  yields 


*0 

J 
— *6 


oC 

dx  g(x)  h(x)  =  -  jdx  f(x)  h'(x)  =  2 


•o 

r 


dx  x~ ^  h' (x)  ,  (5) 


-  OO 


which  is  integrable  for  any  good  function  h(x). 

To  make  sense  of  this  situation,  we  approximate  function  f(x)  by  function 
fc(x)  as  shown  in  figure  1;  that  is,  fE(x)  has  constant  value  f(a+e)  in 
the  neighborhood  e  of  a.  The  approximation  to  generalized  function  g(x)  that 
we  adopt  is 
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9e(x)  -  fj(x)  -  f(a+e)  5 (x-a)  +  f * (x)  U(x-a-e)  ,  (6 

as  depicted  in  figure  2,  where  U(x)  is  the  unit  step  function;  that  is, 
9c(x)  has  an  impulse  of  area  f(a+e)  at  x  -  a,  and  is  equal  to  f'(x)  for 
x  >  a+e.  (We  can  approximate  the  Impulse  as  discussed  at  the  beginning  of 
this  section.) 


Figure  1.  Approximation  f£(x)  to  f(x) 

9E(x) 


area  =  f(a+e) 


Figure  2.  Approximation  g  (x)  to  q ( x ) 
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It  is  important  to  observe  that  if  f(x)  is  singular  (ref.  3,  definition 
21)  at  x  ■  a,  as  for  example  (3),  the  area  f(a+e)  of  the  impulse  at  x  =  a 
becomes  progressively  larger  as  e-*0.  However,  the  area  under  the  remaining 
portion  of  ge(x)  also  becomes  larger,  but  in  such  a  fashion  that  the  total 
area  under  g£(x)  remains  finite  as  e-*0;  this  will  be  shown  below  in  (11). 

To  see  the  effect  of  this  approximation  on  an  integral  of  a  generalized 
function  g(x),  consider  integral 

b 

I  *  J  dx  g(x)  h(x)  .  (7) 

a 

The  approximation  to  I  is: 

b 

Ic  s  j’  dx  g£(x)  h(x) 
a 


b 

*  f(a+e)  h(a)  +  j'  dx  f'(x)  h(x)  ,  (8) 

c 

where  we  employed  (6).  Integration  by  parts  on  (8)  yields  two  alternative 
forms: 


I 

e 


b 

f(a+t )[h(a)  -  h(a+e)]  ♦  f(b)  h(b)  -  ^  dx  f(x)  h'(x) 


a+e 


a+e 

f(a+e)Jh(a)  -  h(a+c)]  +  ^  dx  f(x)  h'(x) 


u 

+  f(b)  h(b)  -  J'  dx  f ( x )  h'(x)  . 


(9A) 
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Since  f ( x )  Is  integrate  at  x  «  a+,  the  leading  term  in  (9A)  and  the  two 
leading  terms  in  (98)  both  approach  zero  as  e-*Q,  yielding  the  limit 

b 

..  n.  .,.f(b,h(b,- .  no, 

£-►0  a 

This  is  the  value  of  (7)  expressed  in  terms  of  integrable  functions. 

The  area  under  approximation  ge(x),  for  c  >  0,  is  available  by 
substituting  h(x)  «  1  in  (8)  and  (9): 

b 

J  dx  gc(x)  -  f (b)  ,  (11) 

a 

which  is  finite  and  independent  of  e.  Thus,  the  impulse  in  figure  2  is 
necessary  in  order  to  compensate  for  the  increasing  area  that  develops  under 
f'(x)  near  x  »  a,  when  e  approaches  zero. 

SINGULARITY  AT  x  *  b- 

Suppose,  instead,  that  f(x)  is  integrable  at  x  *  b-,  but  that  generalized 
function  (2)  is  not.  By  employing  a  procedure  similar  to  that  above,  we  have 
approximation 

9c(x)  *  -f(b-c)  £(x-b)  +  f'(x)  U(b-c-x)  (12) 

to  q(x) .  The  integral  I£  can  then  be  expressed  as 
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b 

I,  -  J  dx  9c(x)  b(x) 
a 


b-c 

»  -f(b-e)  h(b)  +  ^  dx  f'(x)  h(x) 
a 


b-c 

«  -f(b-e)[h(b)  -  h ( b-c 2]  -  f ( a)  h(a)  -  ^  dx  f(x)  h'(x)  .  (13) 

a 

There  follows 


lim  I 

e-*0 


-f (a)  h ( a) 


U 

-I 


dx  f (x)  h' (x) 


(14) 


in  terms  of  integrable  functions. 

The  area  under  approximation  gc(x)  is  finite  and  independent  of  e: 
let  h(x)  *  1  in  (13)  and  get  directly 

b 

£  dx  g£(x)  *  -f (a)  .  (15) 

a 

RELATION  TO  "FINITE  PART"  OF  INTEGRAL 


If  we  start  with  integral  (7)  and  integrate  by  parts,  there  follows 


(16) 


6 


TR  6767 


Now  if  f ( x )  is  singular  at  x  -  a,  the  finite  part  (ref.  3,  page  32)  of  (16)  is 
obtained  by  dropping  the  term  involving  f(a),  thereby  obtaining  identically 
(10).  Conversely,  if  f(x)  is  singular  at  x  ■  b,  the  finite  part  of  (16)  is 
just  (14).  Thus,  the  limit  of  the  approximation  procedure  developed  here  is 
exactly  what  is  yielded  by  the  finite  part  procedure. 
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EXAMPLES 


EXAMPLE  1 


f(x)  »  — --f  for  x  >  0  where  -2  <  v  <  -1  .  (17) 

This  function  is  singular  but  integrable  at  x  *  0.  In  figure  1,  we  identify 

a  -  0  (18) 


and  get 


f ' (x)  ■  xv  for  x  >  0  .  (19) 

Then,  from  (6),  the  approximation  to  the  generalized  function  xv  U(x)  is 

v+1 

gc(x)  =  hn  S(x)  +  x”  U(x'e)  (20) 

and  is  depicted  in  figure  3. 


ge(x) 


8 


Figure  3.  Approximation  to  Genpralized  Function  xv  U(x). 
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The  result  of  applying  (20)  to  a  good  function  h(x)  is  available  from 
(8)  and  (17), 


u  1  u 

j'  dx  g£(x)  h(x)  .  h ( 0 )  +  J  dx  xv  h(x)  , 


(21) 


or  from  either  form  in  (9).  The  limit  follows  from  (10)  and  (7): 


hv+l  Z  yv+l  Z 

1  •£3Th(b)  ‘  ]  dx  h'^X^  *  J  dx  • 


(22) 


The  following  examples  are  derived  in  similar  fashion;  just  the  results 
are  listed. 


EXAMPLE  2 


f ( x )  -  In  x  for  x  >  0 


a  .  0 


f'(x) 


-  for  x  >  0 


9c(x) 


In  e  J(x)  ♦  J  u(x-c) 


U 

I  *  In  b  h(b)  -  J'  dx  In  x  h*(x) 


b 

A  1 
dx  j  h(x) 


(23) 


9 


TR  6767 


EXAMPLE  3 


I 

EXAMPLE  4 


f ( x )  *  ^(ln  x)^  for  x  >  0 
a  -  0 

f *  (x)  -  -5-  for  x  >  0 

9c(x)  -£(ln  c )2S(x)+  Jj-iu(x-c) 

b  b 

*  -|(ln  b)2  Mb)  -  P  dx  ^(ln  x)^  h'(x)  *  dx  h(x)  .  (24) 
0  0 


f(x)  "  £"  x  ■  ^rl  for  x  >  O  ;  v  >  -2 

a  =  0 

f * ( x )  «  xv  In  x  for  x  >  0 


ge(x) 


+  xv  In  X  U(x-e) 


I 


dx  xv  In  x  h ( x )  . 


(25) 
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EXAMPLE  5 

Here  we  consider  the  result  for  the  generalized  function  1/x2  as 
presented  in  the  integral  (1).  Consider  the  ordinary  function 


f£(x)  * 


-  1/x  for  )x)  >  c 

7  sgn(x)  for  |x|  <  e  J 


(26) 


and  its  derivative 


gJx)  -  f ‘ (x) 


1/x^  for  jxj  >  e 
-  f  J(x)  for  |x|  <  c 


(27) 


see  figure  4.  The  function  g  (x)  is  an  approximation  to  generalized  functi 


on 
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l/x^;  it  contains  an  impulse  of  area  -2/e  (at  x  «  0)  that  tends  to  as 

e-*0. 


Thr  approximation  to  integral  (7)  is  as  defined  in  (8);  namely,  for  a  =  -<», 
b  -  +«*, 


-K»  +* 

Ie  -  I*  dx  g€(x)  cos(wx)  -  2  J*  dx  eos^-  -  | 

-•  € 

.  -w  |w|  -  2  1  -  C0^(We)  +  2w  Si (we),  (28) 

where  Si  is  the  sine  integral  (ref.  4,  eq.  5.2.1).  The  limit  as  e-*0  gives 
the  result  for  the  generalized  function  1/x^,  namely. 


f 


-«0 


dx 


— -k  cos(wx) 
yc 


-ir|w|  . 


(29) 


This  result  is  in  agreement  with  ref.  1,  eq.  1-32,  and  with  ref.  3,  page  43, 
for  x-m  with  m  =  2. 


An  alternative  approximation  to  generalized  function  1/x^  that  uses 
finite  functions  is 


gc(x) 


il/x^  for  lxl  >  c 


0  for  lx |  <  e 


-  f  T==r  exp  [-  -^J.  (30) 

e  f2?e  2tdl 


There  follows 
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I 

c 


■MO 


I 


dx  gc(x)  cos(wx) 


2w  Si(wc) 


(31) 


The  limit  as  e-*0  i 
approximations  (27) 
that  develops  about 


s  again  exactly  (29).  The  essential  feature  of 
and  (30)  is  the  large,  sharp  negative  pulse  of  area  -2/e 
x  *  0  as  c  gets  small. 
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APPLICATION  TO  ARRAY  WEIGHTING 


This  example  relies  heavily  on  material  presented  in  ref.  2  on  array 
weighting  for  good  voltage  response  patterns  in  one,  two,  and  three 
dimensions.  In  particular,  from  ref.  2,  eq.  E-7:  for  -2  <  v  <  -1,  we  have, 

for  the  required  array  weighting  w(x)  to  realize  pattern  v+^u^-B ^),  in 

one,  two,  or  three  dimensions  (u  *  -  0,  or  »  the  relation 


-x  w(x) 


for  0  <  x  , 


(32) 


and  zero  otherwise.  To  match  (2),  we  identify 


g(x)  -  -x  w ( x )  , 


(33) 


and 


f(x) 


(34) 


The  function  f ( x )  is  singular  but  integrable  at  x  =  1-,  since  -1  <  \h-1  <  0; 
thus  the  derivative  in  (32)  will  generate  a  generalized  function  (ref.  3,  page  30) 
for  the  weighting  w(x). 


We  now  identify  b  =  1  in  (12)  and  obtain,  via  (34)  and  ref.  4,  eq. 
9.6.28,  the  approximation 


9c(x) 


S(x-l) 


(35) 


Then  there  follows  from  (33),  the  approximate  weighting 


14 


TR  6767 


This  is  the  result  presented  in  ref.  2,  eq.  F-14.  It  is  an  approximation  to 
the  generalized  function 


»<">  -  (ni^)  '.(flP)  U(l-» 


-x)  U ( x )  ,  (37) 

which  is  the  required  array  weighting  according  to  ref.  2,  eq.  E-7. 


We  will  carry  this  example  further  than  the  previous  ones,  by  determining 
the  voltage  response  pattern  that  is  actually  realized  by  an  array  employing 
approximation  weighting  (36)  rather  than  (37).  The  array  voltage  response 
pattern  is  given  by  ref.  2,  eqs.  1-12,  for  any  u,  as 


1 

v ( u )  *  j"  dx  x(j)  Jy(ux)  w( x )  ,  (38) 

0 

where  the  parameter  u  determines  the  array  dimensionality,  and  u  contains  the 
array  geometry,  the  plane-wave  arrival  wavelength,  and  the  various  look  and 
steering  angles.  Substitution  of  generalized  function  (37)  in  (38)  yields 
pattern  (ref.  2,  eq.  E-l) 


v(u) 


for  all  u  , 


where  we  define  entire  function  (ref.  2,  eqs.  17-21) 


(39) 


(40) 
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The  particular  case  of  u+v  -  -1.5  in  (39)  yields  ideal  voltage  pattern 

-  (|fcos(f7^)  -  (4fcosh(lK7)  .  ,41, 

This  relation  is  true  for  all  u,  whether  larger  or  smaller  than  B.  It 

h \'/4 

indicates  a  mainlobe  at  u  ■  0  of  amplitude  (-1  cosh(B),  and  sidelobes  for  u  >  B, 

/2\*  W 

all  of  equal  amplitude  *l-|j  .  Plots  of  this  ideal  pattern  are  available  in 
ref.  2,  figures  13-16. 

When  approximation  we(x)  in  (36)  is  substituted,  instead, in  (38),  we  get 
several  equivalent  expressions  for  the  corresponding  pattern  ve(u),  as  given 
in  appendix  A.  Plots  of  typical  results  are  presented  in  figures  5  and  6  for 
v  »  -1.5.  In  both  figures,  the  curve  labeled  c  *  0  is  the  desired  pattern 
(39).  Figure  5  corresponds  to  a  volumetric-spherical  array  (g  =  .5),  and  the 
desired  pattern  is,  from  (39), 


which  decays  at  a  3  dB/octave  rate  for  large  u.  Figure  6  corresponds  to  a 
planar-circular  array  (u  *  0)  with  desired  pattern  equal  to  ideal  pattern 
(41).  The  approximations  in  both  of  these  figures  for  e  =  .1  are  quite  good, 
but  those  for  c  *  .2  have  undergone  significant  degradation.  The  possibility 
of  replacing  the  delta  function  in  (36)  by  a  narrow  pulse  is  considered  in  the 
next  section. 


Figure  5  furnishes  an  approximation  to  the  bottom  asterisked  case  in 
ref.  2,  figure  12;  figure  6  does  the  same  for  the  middle  asterisked  case  in 
ref.  2,  figure  13.  The  three  asterisked  cases  for  v  =  -1  in  ref.  2,  figures 
11-13,  merely  require  delta  functions  at  x  =  1  and  are  considered  solved.  The 
last  remaining  asterisked  case  is  the  bottom  one  in  ref.  2,  figure  13,  for 
v  =  -2.  But  this  has  already  been  shown  in  ref.  2,  eqs.  E  —37  -  E-39,  to 
involve  a  delta  function  and  its  derivative,  both  of  which  are  easily 
approximated;  see,  for  example,  ref.  1,  pages  279-280,  or  ref.  2,  pages  11-12. 
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Figure  5.  Pattern  of  Spherical  Array  for  Weighting  (36) 


17 


TR  6767 


USE  OF  NEUTRALIZERS 

An  alternative  smoother  approximation  to  f(x)  and  g(x)  than  afforded  by 
figures  1  and  2  is  by  the  use  of  neutralizers;  see  ref.  5,  section  3.3. 
Consider  the  neutralizer  n£(x)  and  its  derivative  shown  in  figure  7;  the 
neutralizer  is  0  at  x  -  a,  and  1  at  x  -  b.  Furthermore,  it  has  derivatives 


n£(x) 


\ 

^  ar..  *  1 


a+2  a+'  b 


Figure  7B.  Function  n^(x) 

Figure  7.  Neutralizer  n£(x)  and  its  Derivative 
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of  all  orders,  all  of  which  are  zero  at  end  points  a  and  b.  The  positive 

parameter  e  characterizes  the  critical  point,  xc  ,  where  the  neutralizer 
is  1/2;  this  point  xc  will  approach  a  as  e-*0+.  The  neutralizer  has 
completed  its  transition  to  1  by  the  value  x  a  a+e.  n  (x)  will  approach  1 
for  all  x  >  a,  as  e-*0. 

The  approximation  we  take  to  f(x)  is  the  product 

fc(x)  s  f (x)  n£(x)  ,  ^43) 

and  the  corresponding  approximation  to  generalized  function  (2)  is  the  smooth 
function 


gc(x)  -  f£'(x)  «  f*(x)  n£(x)  +  f(x)  ne'(x)  .  (44) 

This  approach  is  similar  to  the  regular  sequence  of  good  functions  used  to 
define  a  generalized  function  in  ref.  3,  pages  16-17.  A  representative 
example  is  depicted  in  figure  8.  As  before,  the  area  under  approximation 
9e(x)  is  independent  of  c: 

b  b 

J  dx  gc(x)  .  Jfe(x)J  *  f e (b)  =  f (b)  .  (45) 

The  result  of  applying  gE(x)  to  function  h(x)  is  now 
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Figure  8A.  Function  fE(x) 


Figure  8B.  Function  gc(x) 

Figure  8.  Approximation  to  Generalized  Function  via  Neutralizer 
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I 

c 


f 


dx  g£(x)  h(x) 


i 


dx  fc‘(x)  h(x) 


dx  f c (x)  h * (x) 


b 

-  f(b)  h(b)  -  J 
a 


dx  f(x)  ( x )  h'(x)  , 


(46) 


where  we  have  utilized  the  properties  of  the  neutralizer.  Equation  (46)  now 
replaces  (8).  In  the  limit  of  e-*0,  (46)  yields 


I  .  lim  I 

c 

e-*0 


b 

f (b)  h(b)  -  J  dx  f(x)  h'(x)  , 
a 


in  agreement  with  (10). 


(47) 


A  limiting  case  of  figure  7  is  a  step-function  at  x  =  a*-|;  i.e.. 


n£(x)  .  U(x-a-  J)  . 

Then 

nc ' (x)  =  J(x-a-  J) 


(48) 


(49) 


and  (44)  yields 

gc(x)  =  f'(x)  U(x-a-  |)  +  f(a+|)  $(x-a-  |)  .  (50) 

This  is  almost  identical  to  (6),  which  has  its  impulse  located  at  x  =  a 
rather  than  at  x  =  a+£. 
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APPLICATION  TO  ARRAY 

The  starting  point  is  (32)^  again,  for  the  required  weighting.  The 
function  f ( x )  is  identified  as  in  (34),  and  we  get,  from  (33)  and  (44), 

-xwc(x)  -  g£(x)  -  f'(x)  n£(x)  +  f(x)  n£(x).  (51) 

There  follows,  from  (34)  and  ref.  4,  eq.  9.6.28, 


Substitution  of  (34)  and  (52)  in  (51)  yields,  for  the  approximate  weighting. 


However,  the  neutralizer  in  this  case  must  be  chosen  to  be  1  at  x  =  0, 
and  0  at  x  >  1;  that  is,  it  is  a  reflected  version  of  figure  7A.  For  our 
purposes,  it  is  not  necessary  for  the  neutralizer  to  have  derivatives  of  all 
orders.  Rather,  we  select  n  (x)  so  that  n  (x)  is  continuous  for  all  x, 

•  u  ^ 

and  such  that  n£(x)  and  n£(x)  are  zero  at  the  edges  of  the  transition 
region  (1-e,  1);  see  figure  9.  Here,  letting 


21 


x-l  + 

7~ 


(s*) 
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Figure  9.  Neutralizer  (55)  and  its  Derivatives 


we  take 


n£(x)  =  |  -  |(l5y  -  40y3  ♦  48y5)  for  1-e  <  x  <  1, 


(55) 


and  1  for  0  s  x  j;  1-c.  Then 


nc(x)  =  -  g|^l-8y2+16y4)  for  1-t  j  x  j  1, 


(56) 


and  zero  otherwise.  Also, 


n£(x) 


iO/y_4y3J  for  1-c  j  x  i  1, 


(57) 
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and  zero  otherwise.  From  (55)-(57),  there  follows 


Combining  this  with  the  behavior  of  the  Bessel  function  near  zero 
argument  (ref.  4,  eq.  9.6.7),  we  find  that  both  terms  of  the  approximate 
weighting  w£(x)  in  (53)  are  proportional  to  (l-x)3+v  as  x  approaches  1. 

Thus  if  v  2  -2.  the  approximate  weighting  will  approach  zero  at  least  linearly 
at  x  *  1.  An  example  of  w£(x)  for  v  »  -1.5  and  B  *  4  is  given  in  figure  10 


Figure  10.  Approximate  Weighting  we(x)  for  v  =  -1.5,  B  =  4 
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for  two  different  selections  of  e.  The  large  positive  spike  near  x  =  1  is 
very  pronounced  for  small  c. 

The  array  voltage  response  corresponding  to  approximate  weighting  we(x) 
in  (53)  is  given  by  (38)  as 

v£(u)  - 


where  we  used  (40).  A  program  for  the  evaluation  of  (59)  (along  with 
(53)-(56) )  is  presented  in  appendix  B.  Sample  responses  are  plotted  in  figure 
11  for  a  spherical  array  and  in  figure  12  for  a  circular  array.  Comparison 
with  the  corresponding  results  in  figures  5  and  6  reveals  that  the  impulsive 
weighting  of  (36)  yields  a  better  approximation  to  the  ideal  pattern  (e  =  0) 
than  the  smoother  weighting  of  (53).  That  is,  e  must  be  chosen  smaller  in 
(53)  than  in  (36),  in  order  to  realize  approximately  the  same  voltage  response 
pattern  in  the  first  few  sidelobes. 
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Figure  11.  Pattern  of  Spherical  Array  for  Weighting  (53) 
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SUMMARY 


We  have  indicated  how  approximations  to  some  generalized  functions 
generated  by  the  process  of  differentiation  may  be  realized,  and  then  have 
applied  the  procedure  to  the  approximation  of  the  weighting  required  to 
realize  the  ideal  response  patterns  of  arrays  in  several  dimensions.  Two 
examples,  one  impulsive  and  the  other  smooth,  were  used  for  circular  and 
spherical  arrays,  and  the  resultant  approximate  patterns  were  plotted  for 
different  choices  of  the  parameter  e  governing  the  transition  region  near  the 
singularity  of  the  generalized  function.  How  small  e  must  be  chosen  depends 
on  the  form  of  the  approximate  weighting  and  the  desired  closeness  to  the 
specified  pattern. 


29/30 
Reverse  Blank 
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Appendix  A 

VOLTAGE  RESPONSE  PATTERN  FOR  WEIGHTING  (36) 
With  the  shorthand  notation 


approximate  weighting  (36)  takes  the  form 


for  0  <  x  <  1  , 


(A-l ) 


w£(x)  -  fv+iU-c)  <S(x-l)  +  fv(x)  U(l-c-x)  for  0  <  x  .  ( A— 2 ) 

Substitution  of  (A— 2 )  in  (38)  yields,  upon  use  of  (40),  the  corresponding 
voltage  response 


1-e 

vt(u)  -  fnl(l-c)  ^(u)  ♦  [  dx  x2lrtlj^(ux)  fv(x)  ( A— 3 ) 

0 

for  any  u:  this  is  the  relation  programmed  in  ref.  2,  appendix  F. 

For  small  c  (the  case  of  most  interest),  the  cusp  of  fy(x)  in  (A-3)  at 
x  *  1  causes  numerical  integration  problems.  We  can  alleviate  this  problem  by 
integrating  by  parts  on  (A-3),  using 


U  .  x2“^(ux),  V.-f^fx)  , 


( A— 4 ) 


to  get 


vt(u)  -  -  d-o^w-e))] 

1-e 

♦  ^  dx  x^u~^  j(ux)  fv+^(x)  f°r  u  >  0  •  (A-5) 
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The  function  fy^(x)  is  integrable  at  x  «  1  if  v  >  -2. 

The  completed  integral  over  (0,1)  in  (A— 5 )  can  be  evaluated  in  closed 
form  (ref.  2),  leading  to 


'«<»> 


U4-\H-1\ 


/u2-b2 


1 

-  J  dx  x2u_1  ^_i(ux)  f ( x )  ,  ( A— 6 ) 

1-e 


which  is  advantageous  because  the  interval  (l-e,l)  is  small  for  small  e. 
Finally,  the  cusp  at  x  -  1  can  be  eliminated  for  v  ^  -3/2,  by  changing  the 
variable  of  integration  according  to  x  -  sin  t,  getting 


v 

c 


+  f 


u+1 


-  (l-e)2u^u(u(l-e))| 


r/2 


-  y  dt  (sin  t)2u_1  (cos  t)2'*3  j(u  sin  t)  (J^+1  (8  cos  t)  ,  (A-7) 


where  A  =  arc  sin(l-e),  and 


( A— 8 ) 


Form  (A-7)  is  good  numerically  for  small  e  and  any  value  of  u.  There  is  no 
cusp  at  t  =  it / 2  if  v  2  -3/2.  A  program  for  (A-7)  follows,  where  e,  g,  v,  and 
B  are  arbitrary  (c  >  0,  v  >  -3/2,  B  >  0). 
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l(i  Eps=. 1  €  !  Pattern  Vem>  via  tft-7> 

20  Mu*0  >* 

30  Nu  =  - 1 . 5  & 

40  Be *4  3  !  B  in  (26) 

50  OUTPUT  0;  "Eps  *";Eps:"  Mu  «  ";Mu;’'  Nu  =  "JNu;"  Be  =";E:c 
60  DIM  Ve<0: 480) 

70  COM  U,  Be  ,  Ml ,  N1 ,  M2,  N2 

80  M 1  *Mu- 1 

90  N 1 s  Nu  + 1 

100  M2“2*Mu- 1 

HO  N2*2*Nu+3 

120  ftl pha*Mu+Nu+l 

130  T*2*Eps-Epa-'2 

140  IF  Ep**0  THEM  160 

150  Fle»T'Nl*FNInuxnu<Nl ,  Bc*SC!P<.T)  j  !  *or  line  2  v*’  <fi-7;> 

160  El*l-Ep* 

170  Te=EW2*Mu> 

180  fl*ASN<El> 

190  B=PIx2 

200  FOR  lu*0  TO  480 

210  U»Iu*PI/40 

220  Sq*SQR(ABS<U*U-Bc*Bc )  ) 

230  IF  UOBc  THEN  Tl»FNInu;<nu<R1  pt  a,  Sq>  I  i  ri,  1  of  h  —  7  • 

240  IF  U/Bc  THEN  T 1 *FNJnuxnu < ft  1 pha, Sq> 

250  IF  Ep*>0  THEN  290 

260  T3*T 1 

270  V=0 

238  GOTO  480 

290  T2*FN Jnuxnu <Mu, U> -7« *FN Jnuxnuf Mu, U*E 1  1  1  tor  ,  ne  2  :< 

3O0  T3*T 1 +F 1 e*T2 

310  S»<FNS<A)+FNS<B>>*.5 

320  N-2 

330  H*  <  6-ft ) * . 5 

340  F  *  f  B - ft )  ■ '  3 

350  Vo*9E99 

360  T=0 

370  FOR  K«1  TO  N-l  STEP  2 
380  T*TfFNS<ft+H»K) 

390  NEXT  K 

4O0  S»S  +  T 

410  V  *  <  S  +  T  >  *  F  1  1  l  n  e  3  f  •  n  -  7 

420  IF  ftBS ‘  V-Vo  )  <  *ABS  ('/,•*  1 E-5  THEN  480 

430  Vo*V 

440  H*N*2 

450  H=H*.5 

460  F  =  F  e . 5 

470  GOTO  360 

480  Ve<Iu)  =  T3-V  1  Voltage  R*  if.  :  ns  •:  ft-' 

490  PPINT  Iu.VeUu) 

5O0  NEXT  lu 

510  PLOTTER  IS  “GRAPHICS" 

520  GRAPHICS 

530  SCALE  0,480,-70,0 
540  GRID  40,10 

550  PENIJP 

560  FOR  lu=0  TO  480 

57 O  PLOT  Iu,  20-LGT  •  AES'  Ve  <  Iu  7e  1  0  1  ■  ■ 

580  NE  T  Iu 

590  PENUP 

60O  END 

610  1 
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62G  DEF  FNS<T>  !  int-igrand  of  <8-7 

630  COM  U,Bc,M1,H1,m::,N2 

640  St*3IN(T) 

650  Ct=COS<T> 

660  Tl*FNJnuxnu<Ml , U*St ) 

670  T2*FNInuxnu<Nl , Bc*Ct > 

680  RETURN  St''M2*Ct^N2*Tl*T2 
690  FNEND 
700  ! 

710  DEF  FNGamma<X>  !  GammaOO  via  HART,  page  279,  #5231 
720  N* INT<X> 

730  R  *  X - N 

740  IF  <  N  >0  )  OR  <R<  >0 )  THEN  770 

750  PRINT  "FNGammaGO  IS  NOT  DEFINED  FOR  X  =  '!< 

760  STOP 

770  IF  R/0  THEN  800 
788  Gamma2M 
790  GOTO  840 

800  P*3. 369543591 31  +  R*<  1 . 09850630453+R-K  .  1  ^2  ?2800  7  :■  4r-^F  ♦  :  .  930  i  3464  1 86E- 

310  P*43 . 94 1 0209 1 39  +  R*  <  22 . 968O800836*F>  >  1 2 .  30216981 1  2  +  R  +  P 

820  Q*43. 94102091 91 +  R*<  4. 390564 74596-R*. 7.  1 5075063299-R  > 

830  Camma2*P/Q  !  Gamma  2  +  R  •  fo'*  0  R  1 

340  IF  N >2  THEN  880 

350  IF  N<2  THEN  930 

860  Gamma*Gamma2 

370  GOTO  980 

380  Gamma*Gamma2 

890  FOP  K«1  TO  N-2 

9O0  G  am  m  a*  G  am  m  a#  <  X  -  K > 

910  NEXT  K 

920  GOTO  980 

930  R* 1 

940  FOR  K*0  TO  1-N 
950  R*R*<X+K) 

960  NEXT  K 

9  7  O  G  am  rn  a*  G  am  m  a2  •••'  R 

980  RETURN  Gamma 
990  FNEND 

1000  1 
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1016  DEF  FNJnuxnu<Nu,  X)  !  Jnu(x)/xAt-iu 

1020  IF  flBSOOCl  THEN  1130 

1030  fl*. 797384360803 

1040  IF  Nu*0  THEM  RETURN  FNJoOO 

1050  IF  Nu*.5  THEN  RETURN  fl*SIN<X)/X 

1060  IF  Nu— .5  THEN  RETURN  fl*COS<X> 

1070  IF  Nu*  1  THEN  RETURN  FNJHXJ/X 

1030  IF  Nu—  1  THEN  RETURN  -FNJ1<X>*X 

1090  IF  Nu* 1 . 5  THEN  RETURN  fl*<SIN<X>-X*CQS* X) )  X-3 

1100  IF  Nu*- 1 . 5  THEN  RETURN  -fl*<X*S IN<X > +COS <XJ > 

1110  IF  Nu*2  THEN  RETURN  <2*FNJl  (X)-X*FNJo-'.0 
1120  IF  Nu«-2  THEN  RETURN  <2*FNJ1 (X)-X*FNJovX> >*X 
1130  fl*Nu 

1140  IF  UNT<ft)Oft>  OR  (fl>*0)  THEN  1160 
1150  K*fl  — Nu 

1160  S*T*l//<2/'fi*FNGafnm*<fl+l )  > 

1170  R*- . 25*X*X 

1180  Big*RBS(S) 

1190  FOR  N* 1  TO  100 
1200  T«T*Rx<H*<H+A>> 

1210  S*S+T 

1220  Bi  g*MRX<:.Bi  g,  RBS<S>  > 

1230  IF  RBSCTX-1E-1  1*ABS<.$>  THEN  1270 
1240  NEXT  N 

1250  PRINT  "100  TERMS  IN  FN Jnuxnu < Nu , X >  RT  ;NuJ  , 
1260  PRUSE 

1270  D*12-LGT<RBS(EigxS)>  !  NO.  OF  SIGNIF.  DIGITS 

1280  IF  K  >0  THEN  S*S*<4*RXK 

1290  RETURN  S 

1300  FNEND 

1310  1 

1320  DEF  FN Jo<  X)  !  Jo<X>  via  9.4.1  t.  9.4.3 

1330  V*RBS(X> 

1340  IF  Y >3  THEN  1390 
1350  T*Y*Y '9 

1360  Jo*.  8444479-TX .  0039444-T*.  80021  > 

1378  Jo*  1  -T*  •.  2. 2499997-T *<  1 . 2656208-T* •' .  31 6  3366- T- Jo 
1330  GOTO  1450 
1390  T *  3  Y 

14  00  Jo  =  9. 5 1 2E-5-T* <  .  00137237-T*< . ©0072805-1*. COO  1 4  4 
14  18  Jo*.  797S8456-T-K7. 7E-7  +  T*-.:  .00552740  +  T*  Jo  ■ 

1420  3* .  00262573-T *(. .  808541 25+ T*< .  00029333-T*.  080  3  :5 

1438  T*Y- . 78539316-T*C.841 66397  +  T  * <  3. 954E-5-T  +  3 ■  ■ 
1440  J  o  *  J  o  *  C  0  S  <. T  > ' S  0  R  C  Y ) 

1450  RETURN  Jo 
1468  FNEND 
1470  i 

1480  DEF  F N J 1  <  X  ■'  !  J1<X>  via  9.4.4  t  3.4.6 

1498  Y * R B S <  X  i 

1508  IF  Y >3  THEN  1550 

1518  T * Y*Y  9 

1528  J 1  * .  804433 1 9-T* < . 8803 1761  - T  * . 8880 1  ;  0  9 
I  5 3 8  J  1  * X *  <  .  5 - T *  .  5 6 2 4 9 9 3 5 - T *  .  2 1  8 9 3 5 7  3  -  T  *  ■  .  0  3  *5  42  3 
1548  GOTO  1610 
1558  T=3  Y 

1560  J  1  =  .  o  O  0 1  7 1 85  -  T  -  ■  .  O  8  2  4  9  5 1  1  -  T  *  >'  .  0  8  !  1  3  65  -  T  *  .  0  o  9  ^  j 
1578  J!  =  .  7973S456  +  T*'  1 . 56E-6  + T*  ■  .8165966'  +  ''  -  J  1  1 
1  5'  8  8  S  =  .  O  O  6  3  7  S  7  9  -  T  *  ■'  .  0  0  0  7  4  3  4  &  +  T  *  (  .  8  0  fi  -  •?  £  2  4  -  7  »  .  0  0  8  2  9  . 
1598  T * V - 2 .  3 5 6  1  9 4 4 9  +  T ■* ■.  .  1  24996  1  2*T*  5. 65E-5- T »  9 
1608  J  1  *SGN  V  >  *  J  l  »C  OS  \T  ■  SOP1  V  • 

1610  RETURN  J1 
1620  FNEND 
16  30  1 
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1640  BEF  FNInuxnuCNu,  X )  !  lnu(x)/:;  nu 

1650  IF  ABS<XX1  THEN  17 70 
1660  fl  =  . 398942280401 
1670  E=EXP<X> 

1680  IF  Nu*0  THEN  RETURN  FNIo<X> 

1690  IF  Nu».5  THEN  RETURN  A*<E-l/E>xX 
1700  IF  Nu=- . 5  THEN  RETURN  fl*<E+l/E> 

1710  IF  Nu*  1  THEN  RETURN  FNIKXXX 

1720  IF  Nu»- 1  THEN  RETURN  FNI1<X>*X 

1730  IF  Nu«1.5  THEN  RETURN  fl*((X-l)*E+<X+l)  'E  X -3 

1740  IF  Nu*- 1 . 5  THEN  F'ETURN  fl»<  <X-1  >*E-<X+1  >/£) 

1750  IF  Hu»2  THEN  RETURN  <X*FNIo<X>-2*FNI 1 <X> XX'3 
1760  IF  Nu—2  THEN  RETURN  <X*FNIo<X>-2*FNI  1(X;)»X 
1770  fl=Nu 

1730  IF  <  I  NT  <  AX  >fl)  OR  <A>-0>  THEN  1800 
1790  K*fl*-Nu 

1800  S  =  T«l/<2''fl*FNGafnma<A+l>) 

1810  R* . 25*X*X 
1820  Big-flBS(S) 

1830  FOR  N«1  TO  100 
1840  T=T*R/<N*<N+A>) 

1850  $*S+T 

I860  Bi g=MRX<  B ig,RBS(S>) 

1870  IF  ABS(TX»1E-1 1*ABS^S>  THEN  1910 
1880  NEXT  N 

1890  PRINT  "100  TERMS  IN  FNInuxnuCNu, X>  AT  ";Nu;X 
1900  PRUSE 

1910  D»12-LGT<ABS<Big/S> )  '  NO.  OF  SIGNIF.  DIGITS 

1920  IF  K >0  THEN  S»S*<4*RXK 

1*30  RETURN  S 

1940  FNEND 

1950  ! 

I960  B  E  F  F  N I  o  <  X  >  !  I  o  <  X  >  Mia  9,3.1  l*.  9.3. 2 

1970  Y*ABS 1 X  > 

1930  IF  Y>3. 75  THEN  2030 
1990  T*Y*Y/14. 0625 

20O0  I  o* . 2659732+TX. ©360768+T*. 00458 13) 

201O  Io=l+T*<3. 5 156229  +  T  * <  3. 0899424  +  T*<.  1 . 20*7432+1 +  .o 

2020  GOTO  2O70 
2O30  T-3.75/Y 

2O40  I ©».  009 1 628 1  — TX  .  02057706-T * < .  ©26355 3"  *T*  .  O 1 6 ~ 7-  -  ,  80 292  377  •  •  > 

205O  I  o*  .  39894228  + T*<  .  01  323592>Tim.  .  0O22531  9-7*'  .  001  57-;.  :  T  -  T  -  I •  .•  '• 

2O60  I o= I o*EXP < Y )  SQR<Y) 

2O70  RETURN  lo 
2O80  FNEND 
209O  ! 

21O0  DEF  FNI1-:X>  !  I  1  •  X>  mi  a  9.8.  3  &  9.8.4 
2110  Y*RBS  (  Y  > 

2120  IF  Y  >3 . 75  THEN  2170 
2130  T  ~  Y *  Y  s  14. 0625 

2140  1 1  * .  02653733+T*  < .  O0  30 1 5  32  +  T+ .  O00'324  l  1  • 

2150  1 1  * X *  <  .  5  +  T *  <■  .  8 7 3 9 O 5 9 4  +  T *  < .  5 1  4 9 8 8 6 9  +  T »  •  .  •  5  O  0  4  9  I  ;  +  • 

2160  GOTO  2210 
2170  T  =  3.75-'Y 

2180  1 1  *  .  0 1  03 1  555-T*  <  .  02282967-T  +  ■, .  ©23953  1  2- *  *  .-3.';  .  :  4  -  .  y‘j42&u5  9  >  ‘ 

2  190  I  1  =  .  39394228-T * f  .  ©393302'4-rT -  •.  .  00  3620  IS-’’*  .  00  3*0  1  -  T  - 1  i  >  ■  1 
2200  11=  G N 1  X >*I  1  +  E X P ■  Y  >  SOP'  Y  • 

2210  RETURN  II 
2220  FNEND 
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Appendix  B 

PROGRAM  FOR  THE  EVALUATION  OF  (59) 

The  integral  of  interest  Is,  from  (59), 

1 

*e(u)  -  f  dx^fux)  x2"+1  wc(x)  .  (B-l) 

0 

We  approximate  this  Integral  by  sampling  at  increment  a  -  jjj-  and  using 
Simpson's  rule.  There  follows 


where 


vc(u)  ^  *nA*nuA)  (nA)2w+1  wc(nA)  » 

nlo  0 


( B— 2 ) 


fwn]  *  lj  4’  2’  4’  4*  2»  4»  1  •  (B-3) 

The  program  below  evaluates  (B-2).  Values  of  x2u+1  we(x)  are  stored  in 
array  Xmuwe. 
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1  Ci  Eps*.  1  !  TRANSITION  PARAMETER  € 

26  Mu*0  !  DIMENSIONALITY  PARAMETER  M 

30  Nu— 1.5  !  WEIGHTING  PARAMETER  V 

40  Bc«4  !  WEIGHTING  PARAMETER  B 

50  Nx*2A8  !  NUMBER  OF  INCREMENTS  IN  X 

60  N i  u*240  !  NUMBER  OF  INCREMENTS  IN  l' 

70  DIM  Xmuw#<0: 1024), V*<0:240> 

SO  REDIM  Xmuwe<0: Nx> , V»<0: Ni u) 

90  Mu21*Mu*2+l 

100  Nul*Nu+l 

110  D»1*l'Nx 

120  Xmuw*<Nx>*0 

130  FOR  1-0  TO  Nx-1 

140  X* I *De 1 

150  R2*1-X*X 

160  Rs*SQR<R2) 

170  Br*Bc*Rs 

180  Iv*R2ANu*FNInuxnu<Nu, Br> 

190  IF  X>1 -Eps  THEN  220 

200  Xmuwe<.  I  >*XAMu21*Iv 

210  GOTO  260 

220  Iul*R2ANul*FNInuxnu<Nul ,  Br) 

230  N«*FNN«ut<Eps,X> 

240  N*p*FNN*utp<Eps, X) 

250  Xmuwt<  I  >*XAMu21*<  Iu*Ne-Ivl*N*pxX ) 

260  NEXT  I 

270  FOR  lu*0  TO  Nfu 

280  U*Iu/Ni u*6*PI 

290  Ud*U*D*l 

3O0  T*Xmuw*(0 > +FN Jnuxnu v Mu , U  >  *X«uw*  <  Nx ) 

310  So*S**0 

3‘20  FOR  Ns*  1  TO  Nx-1  STEP  2 

330  So*So+FN Jnuxr>u< Mu,  Ns*Ud>*Xinuwe  Ns  ) 

340  NEXT  Ns 

350  FOR  Ns *2  TO  Nx-2  STEP  2 

360  Se*S*  +  FNJnuxnu1- Mu,  Ns*Ud)*Xwuw«  (Ms- 

370  NEXT  Ns 

380  Ve ( I u ) *T+4*So+2*Se 

390  PRINT  Iu,V*<Iu> 

4O0  NEXT  Iu 

410  PLOTTER  IS  "GRAPHICS" 

420  i G  R  A  F‘  H I C  S 

430  SCALE  0,Niu,-70,0 

4-40  GRID  Niu/6,10 

450  PENIJP 

460  T*20*LGT  <Ve (0) > 

470  FOR  lu=0  TO  Niu 

480  Y=20*LGT 1  ABS<  W.  Iu)  >  >-T 

490  PLOT  I  u ,  Y 

500  NEXT  Iu 

510  PENUP 

520  END 

530  ! 
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540  DEF  FNN«ut<E,X) 

550  IF  X< ■ 1 -E  THEN  RETURN  1 
560  Y»<X-1+.5*E)/E 

570  T«Y*Y 

580  T«15-T*<40--T*43> 

590  RETURN  . 5-.l25*Y#T 
600  FNEND 
610  ! 

620  DEF  FNN«utp(E, X) 

630  IF  X< *  1  -E  THEN  RETURN  0 
640  Y»<X-1+.5*E)/E 

650  T«Y*Y 
660  T»1-T*<3-16*T> 

670  RETURN  -1.8?5*T/E 
6S0  FNEND 
690  ! 


ALL  OTHER  FUNCTIONS  FIRE.  LISTED  IN  APPENDIX  A. 
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PERFORMANCE  OF  THREE  AVERAGING  METHODS, 
FOR  VARIOUS  DISTRIBUTIONS 

by 

Albert  H.  Nuttall 
Naval  Underwater  Systems  Center 
New  London,  CT  06320  USA 


ABSTRACT 


The  performance  of  three  averaging  methods,  namely  the  sample  median,  the 
sample  arithmetic  mean,  and  the  sample  geometric  mean,  are  analyzed  In  terms 
of  their  bias,  variance,  and  mean  square  error.  The  bias  and  variance  are 
numerically  evaluated  for  various  parent  distributions  and  plotted  versus  the 
number,  N,  of  data  points  employed  In  the  sample  statistics.  Also,  the 
limiting  behaviors,  as  N  Increases  without  limit,  are  derived.  It  Is  found 
that  the  best  averaging  method  Is  very  dependent  upon  the  distribution  of  the 
data,  with  the  sample  median  being  favored  for  data  with  occasional  large 
out-1  lers. 


INTRODUCTION 


Estimation  of  average  properties,  such  as  the  average  power  In  a  particular 
angular  sector  and/or  frequency  bln  Is  often  accomplished  by  taking  N 
Independent  measurements  of  such  data  and  calculating  a  simple  arithmetic 
average.  However,  when  the  desired  process  Is  subject  to  random  fade-outs  or 
occasional  large  out-1  lers,  this  sample  arithmetic  mean  (SAM)  is  severely 
perturbed,  and  alternative  averaging  methods  should  be  considered.  Two 
possible  candidates  are  the  sample  median  (SMD)  and  the  sample  geometric  mean 
(SUM);  these  nonlinear  processors  of  the  available  data  have  the  potential  of 
suppressing  the  deleterious  effects  mentioned  above.  Here  we  Investigate*  the 
performance  of  all  three  of  these  averaging  methods  in  terms  of  the  number,  N, 
of  independent  data  points  employed  In  the  pertinent  average,  and  the  parent 
distribution  of  the  data.  A  wide  variety  of  distributions  are  considered, 
some  with  parameters  which  allow  for  significantly  different  character  and 
shapes  of  the  governing  probability  functions. 


*The  basic  analysis,  derivations,  and  programs  are  given  in  Ref.  1. 
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DEFINITIONS 


We  have  available  N  statistical ly-lndependent  Identically-distributed  samples 
(random  variables)  xi,  x?,  . xw  from  some  parent  population  with 
cunulatlve  distribution  function  P(u)  ■  Prob{x<u>  and  probability  density 
function,  PDF,  p(u)  ■  P'(u).  The  SAM  of  the  available  measurements  Is 

a(N)  *  ^  (xi  +  x2  +  +  xn)‘  (1) 

the  SMD  Is 

qlN)  ■  middle  value  of  {x^,  x2,  ....  xN],  for  N  odd;  (2) 

and  the  SUM  Is  (for  non-negative  random  variables) 

/  \l/N  /l n (x.>  . •«  +  In 

g(N)  «  (x1  x g  ...  xN)  '  *  exp  I - U - 


for  any  base  B  >  0 
and  scaling  A. 


A  logg(xj}+  ...  +  A  1  ogB  (x^) 
B  ^  "  : 


The  last  form  In  (3)  for  base  B  *  10  goes  under  the  name  of  dB  averaging. 

As  N  tends  to  Infinity,  the  sample  quantities  above  tend  to  definite 
(non-random)  limits.  In  particular,  as  N-*  —  , 


a(N)-* 

q(N)— 

g(N)-* 


arithmetic  mean  »Jdu  u  p(u); 
median  *  u^2  »  where  Pful/2^  *7  » 


geometric  mean 


(4) 


where  we  drop  the  prefix  'sample'  for  these  deterministic  quantities.  The 
last  result  In  (4)  follows  from  the  exponential  form  of  the  SGM  in  (3).  If 
the  1/2  In  the  median  definition  Is  replaced  by  r,  we  have  for  q(N)  the  sample 
quantile  of  order  r  (Ref.  2,  page  181). 

The  three  limiting  quantities  In  (4)  will  generally  not  be  equal.  For 
example,  for  an  exponential  parent  PDF 

p(u)  »  jjj  exp^-  j-j)  for  u  >  0,  (5) 
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we  have 


arithmetic  mean  ■  m; 
median  *  m  In (2)  »  m  .693; 
geometric  mean  ■  me-*  ■  m  .562. 


(6) 


Thus  we  define  the  bias  of  each  of  the  sample  statistics  (l)-(3)  as  the 
difference  between  their  mean  value  and  their  asymptotic  value: 


bias  {SAM}  ■  aHT)  -  mean; 
bias  {SMD-)  ■  qTHT  -  median; 

bias  {SGM]  ■  g(N)  -  geometric  mean.  (7) 

By  virtue  of  tills  definition,  all  three  biases  will  tend  to  zero  as  N-**o  ; 
that  Is,  all  three  estimators,  (l)-(3),  are  asymptotically  unbiased,  each  with 
respect  to  Its  desired  value  as  given  by  (4),  respectively. 

It  Is  then  convenient  to  define  a  normalized  bias,  NB,  for  each  sample 
statistic  as 


NB(N)  •  N  (8) 

O 

where  o  Is  the  standard  deviation  of  parent  PDF  p(u).  The  scale  factor  of  N 
leads  to  a  non-zero  value  of  the  normalized  bias  for  large  N,  while  the  scale 
factor  of  o  Is  convenient  In  that  It  eliminates  the  dependence  of  the 
normalized  bias  on  the  absolute  scale  of  the  Input  data.  For  large  N,  (8) 
yields 


bias  -  o 


NB{«  ) 
R — 


as  N  -*«•  ; 


(9) 


thus  NB(<*>)  Is  an  Important  measure  of  quality  of  the  particular  sample 
statistic  under  consideration. 

The  variances  of  sample  statistics  (l)-(3)  are  defined  as 


2 


» 

f 

i. 

<0 

> 

a2(N)  -  a ( N )  ; 

var{SM0]  ■ 

2 

q^N)  -  q(N }  ; 

var{S(jM"l  * 

2 

g2(N)  -  g(N)  . 

(10) 
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Again,  since  these  quantities  tend  to  zero  for  large  N,  It  Is  more  convenient 
to  define  a  normalized  variance,  NV,  as 


NV( N)  -  N  ,v,arj,ance  .  (11) 


Then  we  can  state  that 

variance  -  a2  — as  N— »  ;  (12) 


thus  NY(«e  )  Is  also  an  Important  measure  of  the  quality  of  a  particular  sample 
statistic. 

We  present  results  here  for  NB ( N )  and  NV(N),  along  with  their  asymptotic 
values  at  N  *  »  ,  for  a  variety  of  parent  distributions  P(u).  Additional 
results  for  the  sample  quantile  with  r  ■  .75  and  .9,  and  for  the  PDF, 
cumulative  distribution  function,  characteristic  function,  cumulants,  and 
moments  of  the  various  sample  statistics  are  available  In  Ref.  1. 

RESULTS 


The  first  case  we  consider  Is  the  Gaussian  PDF  with  mean  m  and  variance  a2. 
Since  this  random  variable  can  go  negative,  the  SGM  Is  undefined.  The  SAM  and 
SMD  are  unbiased  for  all  N;  thus  NB(N)  «  0  for  all  N,  for  this  example. 

Results  for  the  normalized  variance  are  presented  In  Fig.  1  for  the  number  of 
samples,  N,  between  1  and  51,  for  both  the  SAM  and  the  SMD.  The  normalized 
variance  for  the  SMD  Is  computed  only  at  odd  values  of  N,  Indicated  by  an  X, 
and  straight  lines  drawn  between  these  points  for  ease  of  association  of 
values.  It  Is  seen  that  the  variance  of  the  SMD  Is  always  greater  than  that 
for  the  SAM,  the  limiting  value,  NV(oo  ),  being  »/2  for  the  SMD;  see  also  Ref. 
2,  page  369.  Observe  that  the  parameters  m  and  o  of  this  PDF  have  dropped  out 
of  this  normalized  plot. 

It  Is  worth  pointing  out  here,  and  for  similar  results  to  follow,  that 
although  the  curve  for  the  SMD  Increases  with  N,  that  does  not  mean  that  the 
variance  Increases  with  N;  rather,  the  normalizing  factor  of  N  In  definition 
(11)  causes  this  behavior.  The  actual  (unnormal  1  zed)  variance  decreases 
monotonlcally  with  N,  eventually  being  of  order  1/N. 

For  a  Rayleigh  random  variable,  the  results  for  the  normalized  bias  are  given 
In  Fig.  2.  The  SAM  Is  unbiased  for  all  N,  whereas  the  SGM  and  SMD  are,  of 
course,  only  asymptotically  unbiased.  The  limiting  values,  NB(*>),  for  both 
of  these  latter  sample  statistics  are  given  by  analytically  complicated 
expressions  and  are  not  repeated  here,  for  the  sake  of  brevity;  they  are 
Indicated  numerically  by  horizontal  lines  at  the  right  edge  of  the  figure. 

The  corresponding  results  for  the  normalized  variance  are  given  in  Fig.  3. 

They  Indicate  that  whereas  the  SGM  has  about  the  same  stability  as  the  SAM, 
the  variance  for  the  SMD  Is  about  651  greater. 

For  an  exponential  PDF  (as  given  by  (5)),  the  normalized  bias  and  variance 
results  are  presented  In  Figs.  4  and  5  respectively.  The  biases  of  the  SGM 
and  SMD  are  comparable,  but  we  observe  that  the  variance  for  the  SGM  is  twice 
as  small  as  that  for  the  SAM  and  the  SMD. 
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For  a  log-normal  PDF, 


(13) 


the  Nb(N)  and  NV(N)  results  are  Independent  of  location  parameter  nty,  but 
they  do  depend  on  spread  factor  oy.  This  may  be  anticipated  by  plotting  the 
PDF  (13)  for  various  values  of  oy  and  observing  that  the  shape  changes  as 
oy  does.  Since  NB(N )  and  NV(N)  depend  upon  the  shape  of  the  PDF  (rather 
tnan  upon  absolute  location  and  scale),  results  will  depend  on  the  particular 
value  of  oy  selected.  An  example  of  NB(N)  and  NV( N )  for  oy  »  1  Is 
presented  In  Figs.  6  and  7.  Now  we  observe  that  the  variance  of  the  SMD  Is  3 
times  better,  and  that  of  the  SGM  4.6  times  better,  than  for  the  SAM,  at  least 
for  larger  values  of  N.  However,  as  oy-»0,  the  log-normal  PDF  in  (13) 
approaches  a  Gaussian  PDF  about  the  point  u  *  exp(m),  and  the  behaviors  would 
revert  back  to  Fig.  I  then. 

The  next  example  Is  the  Rice  PDF;  physically,  this  corresponds  to  the 
squared-envelope  of  the  sum*  of  a  sine  wave  and  a  centered  narrowband  Gaussian 
noise  process.  That  Is,  the  PDF  Is 

p(u)  *  exp(*  JL"2~0  Io^r*9for  U  >  0  *  (14) 

where  A  Is  the  sine  wave  amplitude  and  a;  Is  the  noise  standard  deviation. 

Once  again,  the  shape  of  the  PDF  depends  on  a  parameter,  namely  A/cr,  .  Results 
for  A/<r,  ■  1  are  given  In  Figs.  8  and  9.  The  SMD  has  14^  greater  variance 
than  the  SAM,  but  the  SGM  has  about  60%  of  the  SAM  variance.  As  A/oj—0,  the 
exponential  PDF  results  are  obtained,  whereas  as  A/a;  •*«*,  the  Gaussian  case  is 
realized.  Thus  (14)  represents  a  transition  case  between  these  extremes. 

The  last  example  we  consider  here  Is  an  exponential  PDF  with  out-1 lers.  That 
is,  each  sample  or  measurement  In  ( 1 ) - ( 3)  is  given  by 

*lc  *  xa  +  xb,  (15) 

where  xa  has  an  exponential  PDF, 

Pa(u)  *  jjp  exp  for  u  >  0,  (16) 


*If  each  of  the  observed  random  variables  xj,  X2,  ...,  xn  in  { 1 ) - ( 3 )  Is 
obtained  by  first  summing  up  the  envelope-squared  outputs  of  M  narrowband 
filters,  as  for  example  in  diversity  reception,  the  PDF  in  (14)  is  replaced  by 
the  distribution.  Results  for  this  case  are  available  in  Ref.  1,  but  are 
not  given  here,  for  sake  of  brevity. 
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and  disturbance  xj,  Is  a  random  variable  which  Is  zero  most  of  the  time,  but 
occasionally  takes  on  a  large  value  (out-1  ler)  L.  That  Is,  Its  PDF  is 

Pb(u)  -  (1  -  Q)  £(u)  +  Q  &(u  -  l),  (17j 

where  Q  Is  the  probability  of  an  out-1 ler.  Then  the  parent  PDF  of  observation 
(sample)  Is  the  convolution  of  (16)  and  (17): 

P<“>  ■  -HT2*'0  6r)  “(“>  (-TT1)  <181 

d  &  d  \  d  ' 

where  unit  step 


The  Important  parameter  now  Is  L/ma,  which  obviously  affects  the  shape  of 
PDF  (18). 

Now  however,  before  we  get  Into  the  detailed  bias  and  variance  results, 
another  consideration  Is  of  paramount  Importance.  Our  sample  statistics, 
(l)-(3),  will  no  longer  extract  (estimate)  the  arithmetic  mean,  median,  and 
geometric  mean,  respectively,  of  the  (disturbance-free)  exponential  PDF  (16), 
but  perforce,  the  corresponding  statistics  cf  the  measurement  PDF  (18).  If, 
however,  we  are  really  Interested  In  the  parameters  of  (16),  then  we  must 
Inquire  Into  the  quantitative  disturbance  caused  by  the  out-llers  described  in 
(17).  Here  we  merely  cite  the  results  for  one  numerical  case;  additional 
results  are  given  In  Ref.  1. 

For  probability  Q  •  .05,  and  out-1 ler  value  L/ma  »  6,  we  find  that  the  ratio 
of  means,  for  (18)  with  respect  to  (16),  Is  1.3.  The  corresponding  ratio  of 
medians  Is  only  1.08,  whereas  the  ratio  of  geometric  means  Is  1.13.  Thus  the 
SMD  and  SGM  are  more  resistant  to  the  presence  of  Infrequent  out-llers. 

Insofar  as  their  effects  on  the  particular  parameters  of  median  and  geometric 
mean. 

The  results  for  the  normalized  bias  and  variance  are  given  In  Figs.  10  and 
11.  The  variances  of  the  SMD  and  SOM  are  again  smaller  than  that  for  the 
SAM.  The  bias  of  the  SMD  and  SOM  are  comparable. 

If  we  define  a  mean-square  error  as  the  average  value  of  the  squared 
difference  between  a  sample  statistic  s  and  a  desired  parameter  da  of  the 
disturbance-free  PDF,  we  can  develop  It  as  follows: 


MSE  »  (s  -  day  -  (s  -  s  ♦  5  -  day 


♦ 


(20) 
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But  the  first  term  Is  the  variance  of  the  sample  statistic,  and  the  second 
term  can  be  expressed  as 

s  -  da  ■  (s  -  d)  +  (d  -  da)  »  bias  +  deflection  in  desired  parameter,  (21) 


where  d  Is  the  modified  value  of  the  desired  parameter  da,  due  to  the 
disturbance.  Thus 

MSE  *  variance  +  (bias  +  deflection)^  .  (22) 

Now  the  bias  and  variance  are  0(N~1)  for  large  N,  whereas  the  deflection  of 
the  desired  parameter  does  not  decay  with  N  at  all;  In  fact.  It  Is  Independent 
of  N.  Thus  the  considerations  above,  whether  for  the  ratio  of  means  or 
medians  or  geometric  means,  are  very  Important,  since  they  dominate  the 
magnitude  of  the  mean-square  error  for  very  many  samples  available. 

DISCUSSION 


The  ability  of  the  SMD  and  SGM  to  suppress  deleterious  effects  due  to 
occasional  large  Interferences  Is  very  pronounced  for  some  probability  density 
functions.  Not  only  Is  the  deflection  of  the  desired  parameter  (mean  or 
median  or  geometric  mean)  decreased,  but  the  bias  and  variance  of  the  estimate 
can  be  markedly  reduced  In  some  cases.  The  exact  amounts  depend  on  the 
magnitude  and  frequency  of  the  Interference. 

Another  possible  approach  to  alleviate  the  effects  of  additive  large  out-1  lers 
Is  to  subject  the  available  samples  xi,  x?,  ...»  xu  to  a  nonlinear 
transformation  such  as  saturation.  In  order  to  suppress  the  large  contri¬ 
butions,  prior  to  evaluating  the  SAM  or  SGM  or  SMD.  Knowledge  of  relative 
levels  (such  as  L/ma  for  the  above  example)  would  be  required  for  optimal 
adjustment  of  the  saturation  level,  but  performance  could  be  markedly  Improved. 
The  nonlinear  transformation  would  reduce  the  deflection,  while  the  averaging 
of  N  samples  would  reduce  the  bias  and  variance.  This  possibility  has  not  yet 
been  pursued. 
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Detection  Performance 
Characteristics  For  a  System 
With  Quantizers,  Or-ing, 
And  Accumulator 


A.  H.  Nuttall 

ABSTRACT 

The  false  alarm  and  detection  probabilities  of  an  N-channel  system 
subject  to  L+ 1  level  quantization,  or-ing,  and  accumulation  of  M  time 
samples,  are  evaluated  exactly,  with  no  Gaussian  assumptions,  for 
arbitrary  values  of  N,  L,  M,  and  for  a  quantizer  with  arbitrary  break¬ 
point  locations.  The  channel  noises  are  Independent  but  can  have 
arbitrary  statistics.  The  signal  occupies  one  unknown  (possibly 
changing)  channel,  if  present.  Two  FFTs  suffice  to  sweep  out  a 
complete  detection  probability  vs.  threshold  curve.  The  optimum 
placement  of  quantizer  breakpoints,  for  a  fixed  total  number  of  levels, 
L+ 1,  is  a  subtle  one  and  is  shown  to  depend  on  N,  M,  and  the  desired 
level  of  performance;  a  simple  rule-of-thumb  is  presented  which 
yields  near-optimum  capability  over  the  useful  range  of  the  operating 
characteristics.  A  brief  comparison  with  the  optimum  processor  is 
made. 
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DETECTION  PERFORMANCE  CHARACTERISTICS  FOR  A 
SYSTEM  WITH  QUANTIZERS,  OR-ING,  ANO  ACCUMULATOR 

INTRODUCTION 

Practical  realizations  of  desired  systems  often  incorporate  approximations 
to  the  ideal  processor  or  devices,  for  the  sake  of  reduced  expense  and 
equipment  complexity.  In  particular,  quantization  is  frequently  employed, 
since  it  facilitates  data  handling  and  processing  in  terms  of  storage  and 
execution  time.  In  addition,  tne  large  number  of  alternatives  for  signal 
presence  and  location  that  must  be  considered  often  dictates  that  data 
reduction  procedures,  sucn  as  or-ing,  be  adopted.  Both  of  these  suboptimum 
approaches,  quantization  and  or-ing,  degrade  system  performance,  and  it  is 
important  to  know  tne  extent  of  the  degradation.  Alternatively,  it  is 
desirable  to  know  how  much  the  received  signal  strength  must  be  increased  in 
order  to  maintain  a  specified  level  of  performance. 

The  effects  of  or-ing  by  itself  were  analyzed  in  ref.  1,  where  the 
required  input  signal-to-noise  ratio  for  specified  false  alarm  and  detection 
probabilities  was  evaluated  as  a  function  of  tne  number  of  input  channels  and 
tne  observation  time.  In  ref.  2,  the  additional  degradation  caused  by  the 
inclusion  of  quantization  was  derived,  where  the  number  of  levels  and 
breakpoint  locations  of  the  quantizer  were  completely  arbitrary.  Both  of 
these  analyses,  however,  were  limited  to  second-order  moments  of  the  decision 
variable,  and  were  therefore  most  appropriate  to  the  situation  of  large 
observation  time  and  moderate  false  alarm  probabilities.  That  is,  the 
Gaussian  presumption  played  a  prevalent  role  in  the  analysis.  A  cursory  study 
of  good  quantizer  breakpoint  locations  was  also  given  in  ref.  2. 

Numerical  evaluation  of  tne  or-ing  losses,  based  upon  the  derivations  in 
refs.  1  and  2  ,  were  given  in  refs.  3  and  4,  and  a  more  extensive  investiga¬ 
tion  of  quantizer  characteristics  was  conducted  in  ref.  5,  which  corroborated 
the  results  given  in  ref.  2.  But  again,  all  results  were  based  on  a  second- 
order  moment  approach. 
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Here  we  will  derive  exact  results  for  the  faise  alarm  and  detection 
probabilities  of  a  system  with  quantization,  or-ing,  and  accumulation-in-time, 
for  arbitrary  input  signal  and  noise  statistics.  Surprisingly,  it  turns  out 
that  inclusion  of  the  additional  nonlinearity  (quantization)  actually 
simplifies  tne  analysis,  and  the  judicious  use  of  FFTs  (fast  Fourier 
transforms)  makes  tne  numerical  evaluation  of  the  probabilities  an  efficient 
and  accurate  procedure. 
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SYSTEM  DESCRIPTION  ANO  ASSUMPTIONS 

The  system  of  interest  is  depicted  in  figure  1.  There  are  N  input 
channels,  each  of  which  is  subject  to  noise  statistically  independent  of  the 
other  channel  noises.  A  signal  is  either  present  on  one  (and  only  one) 
unknown  input  channel  or  it  is  not  present  at  all;  we  wish  to  make  a  decision 
on  presence  versus  absence  with  good  quality.  The  input  signal  (if  present) 
need  not  be  additive  to  the  input  noise;  all  that  is  required  is  knowledge  of 
the  probability  density  function  of  the  input  channel  random  variable  xn(m) 

(at  sample  time  m)  for  signal  present.  By  setting  the  signal  strength  to 
zero,  we  ootain  the  probability  density  function  for  noise-only,  of  course. 

It  is  presumed  that  the  signal,  if  present,  remains  so  for  all  M  discrete 
time  samples  accumulated  at  the  or-ing  output.  However,  tne  signal  could 
remain  in  one  channel,  or  it  could  wander  over  any  number  of  channels  in  a 
deterministic  or  random  manner,  from  time  sample  to  time  sample. 

The  proDability  density  functions  of  inputs  {xn(mj}  are  arbitrary  except 
that  they  must  be  identical  for  all  the  noise-only  channels.  Extension  to 
non-identical ly  distributed  channel  noises  appears  possible  but  nas  not  been 
pursued.  No  Gaussian  assumptions  are  made  at  any  point  of  the  system  of 
figure  1,  for  the  general  analysis  to  follow.  However,  tne  numerical  example 
that  we  eventually  pursue  is  for  Gaussian  inputs,  although  this  could  easily 
be  replaced  by  a  different  case  of  interest;  in  fact,  some  candidate  examples 
are  listed  later  in  an  appendix. 

Each  channel  input  xn(m)  is  subject  to  memoryless  quantization,  where 
tie  quantizer  output  q{xn(m)}  nas  a  total  of  L+1  levels,  and  the  L  quantizer 
abscissa  breakpoints  are  arbitrary;  see  figure  2.  A  general  analysis  for  this 
iiost-general  quantizer  is  possible  and  will  oe  outlined  later.  We  confine  our 
attention  here  to  the  less-general  quantizer  depicted  in  figure  3,  where  the 
ordinate  (output)  levels  are  limited  to  tne  equi-spaced  values  0,  1,  2,  ...,  L, 
but  the  abscissa  (input)  breakpoints  are  arbitrary,  except  that 
-  ^+1  ^or  *  £  <  L-l,  without  loss  of  generality. 
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Figure  1.  Nonlinear  System 


Figure  2.  General  Quantizer  Characteristic 


Figure  3.  Specific  Quantizer  Characteristic 
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This  limitation  on  the  possible  quantizer  output  levels  is  not  as 
restrictive  as  first  appears.  Reference  to  figure  1  quickly  reveals  that  if 
the  quantizer  output  levels  were  at  equi-spaced  values  hQ  +^ah,  for 
0  £  JL  <  L,  the  operating  characteristics,  namely  detection  probability  versus 
false  alarm  probability,  would  be  completely  independent  of  hQ  and 
an  (>  0).  Tnat  is,  hQ  and  an  affect  the  absolute  scale  of  y(m)  and  z,  but 
they  can  be  absorbed  in  a  modified  threshold  at  the  system  output.  In  fact, 
if  one  wants  to  approximate  a  given  nonlinearity  (such  as  tanh(x),  for  example) 
by  a  quantizer,  L  can  be  selected  large  enough,  hQ  and  ah  chosen  without 

restriction,  and  the  h)  q  selected  for  a  good  fit.  Then  the  analysis,  as  con¬ 
tained  here  for  the  quantizer  of  figure  3,  applies  directly,  where  hQ  and  ah 
(determined  from  tne  nonlinear  fit)  are  discarded.  hQ  and  ah  are 
temporarily  used  for  the  fitting  procedure,  but  are  not  fundamental  to  the 
system  operating  characteristics. 

An  example  of  a  fit  to  tann(x)  is  given  in  figure  4  for  L  =  7,  where  the 


Figure  4.  A  Quantizer  Fit  to  tanh(x)  for  L  =  7 
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maximum  ordinate  error  has  been  minimized  and  equalized  at  all  the  steps,  and 
both  functions  approach  *1  as  x  tends  to  ±®.  The  best  locations  of  the 
breakpoints  are  at 


for  1  <  Jl  <  L 


(1) 


for  this  example.  We  also  have  hQ  -  -1,  ah  ■  2/L. 


The  or-ing  device  in  figure  1  is  subject  to  N  inputs  at  each  discrete  time 
sample  m.  It  selects  the  largest  of  these  N  random  variables  at  each  m  and 
emits  it  to  the  accumulator  as  variable  y(m) . 


The  accumulator  adds  up  M  input  time  samples  to  yield  the  decision  variaDle 

M 

z  -  y(m)  .  (2) 

m-1 

This  output  z  is  compared  with  a  threshold.  If  the  threshold  is  exceeded,  a 
signal  is  declared  present  at  the  input  to  the  system;  otherwise,  no  signal  is 
declared.  For  present  analysis  purposes,  it  is  presumed  that  the  discrete 
time  samples  (which  can  actually  take  place  on  the  input  channels)  are 
sufficiently  separated  in  time  that  the  M  random  variables  entering  the 
accumulator  are  statistically  independent. 


When  quantizer  q  in  figure  1  is  monotonical ly  nondecreasing,  the  non¬ 
linear  system  in  figure  1  is  equivalent  (for  all  inputs)  to  that  shown  in 
figure  5,  where  the  quantization  and  or-ing  operations  have  been  interchanged.* 
The  random  variable  y(m)  in  figure  5  is  identical  to  that  in  figure  1.  We  have 
added  another  random  variable,  w(m) ,  in  figure  5  that  has  no  counterpart  in 
figure  1,  for  purposes  of  analysis.  This  interchange  of  operations  is  valid 
whether  the  quantizer  of  figures  2  or  3  is  used. 

Extensions  of  the  above  assumptions  to  more  general  situations,  such  as 
statistically  dependent  inputs  or  dependent  accumulator  samples,  are  discussed 
in  ref.  2.  However,  the  analysis  there  is  limited  to  second-order  moments, 
not  threshold-crossing  probabilities  as  considered  here. 


*See  refs.  2  and  4  for  two  different  proofs. 
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ANALYSIS  OF  PERFORMANCE 

In  tnis  section,  we  evaluate  tne  false  alarm  and  detection  probabilities 
of  the  system  in  figure  5.  The  cumulative  distribution  function  of  an 

individual  channel  input,  for  signal  absent,  is  P^,  where 

P(J}(u)  *  Probability  (x  <  u ) signal  absent)  .  (3) 

Tne  cumulative  distribution  function  of  the  channel  input  with  signal-present 

is  P^.  Both  of  these  cumulative  distribution  functions  are  presumed  known. 

The  superscript  will  denote  either  hypothesis  HQ  (no  signal  present)  or 
hypotnesis  (signal  present  on, that  channel).  Notice  there  is  no 

restriction  on  the  forms  of  P^  or  P^;  thus  arbitrary  input  statistics 

(including  nonadditive  signals)  are  allowed.  However,  the  noise-only  channels 
are  identically  distributed. 

Since  tne  or-ing  output  in  figure  5  is 

w(m)  *  maxjx^m),  ...,  xN(m)}  for  1  <  m  s  M  ,  (4) 

tne  cumulative  distribution  function  of  w(m)  is,  for  any  time  instant  m, 


since  the  signal  (if  present)  is  in  only  one  (arbitrary)  input  channel.  In 
fact,  tne  occupied  channel  can  change  from  sample  m  to  m+1,  in  a  random  or 
deterministic  fashion,  and  (5)  is  still  applicable. 

The  output  y(m)  of  the  quantizer  in  figure  5  is  limited  to  the  L+l  values 
X  =  0,  1,  2,  ....  L;  see  figure  3.  Therefore  the  probability  density  function 
of  random  variable  y(m)  is  impulsive,  as  shown  in  figure  6,  where  the  area 
of  the  impulse  at  X  is  given  by 
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for  X -  0 
for  1  s,  X  i 
for  4-  L 


(6) 


Here  jbjjL  are  the  L  abscissa  breakpoints  of  the  quantizer,  and  Pw  is  the  cumu¬ 
lative  distribution  function  of  w,  given  by  (5). 


The  output  of  the  accumulator  in  figure  5  is  called  the  decision  variable 
and  is  given  Dy  the  sum  of  M  statistically  independent  random  variables 
according  to 

M 

z  y(m)  .  (7) 

m=l 


Since  y(m)  can  only  take  on  the  values  0,  1,  2,  ....  L,  the  decision  variable 
z  can  only  take  on  the  values  0,  1,  2,  ML.  Thus  the  probability  density 
function  of  z  is  also  impulsive,  as  depicted  in  figure  7;  the  area  of  the 
impulse  at  n  is  denoted  by  an. 


We  now  need  to  relate  the  {an]  of  figure  7  to  the  [a^J  of  figure  6.  To 
accomplish  this,  we  resort  to  the  characteristic  functions  of  y(m)  and  z. 
Using  tne  statistical  independence  of  the  [y(m)}  ,  the  characteristic  function 
of  z  is 

M  M 

fz(f)  =  exp(i?z)  =  jexp(iTy)"^  =  jfy  ($)j  ,  (8) 


where  f  is  the  characteristic  function  of  random  variable  y(m).  But  by  use 
of  figures  6  and  7,  we  have 

*o  L 

fy($)  =  J du  Py(u)  exp(iju)  !2!(  exP(i5i)  >  (9) 

Jl»0 

and 


-•© 


oo  Ml 

f£)  «  P2(u)  exp(i$u)  =  ^  Bn  exp(ijFn) 

n=0 


(10) 
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Substitution  of  (9)  and  (10)  in  (8)  yields 


ML 

V?)  «  5  Bn  exp(ifn)  - 
n«0 


(ID 


Now  the  results  of  appendix  A  indicate  that  the  set  of  numbers  can 

<  nj0 

be  recovered  directly  oy  an  Nf-point  OFT  (discrete  Fourier  transform)  of  the 
set  of  characteristic  function  samples 


f2(2irn/Nf)  for  0  <  n  $  Npl 


(12) 


provided  that 

Nf  2  ML+1 


(13) 


And  from  (11),  we  have  the  required  characteristic  function  samples  as 


fz(2»n/Nf)  = 


M 


exp( i  2wn^/Nf ) 


(14) 


The  sum  on  X  here  is  the  conjugate  of  an  Nf-point  OFT  of  the  t+1  real 
nonzero  numbers  fa/l L  augmented  by  zeros.  When  this  DFT  is  raised  to  the  M-th 

l  X>  g 


power,  it  constitutes  the  required  Nf  samples  of  the  characteristic  function 
f2  that  are  needed  for  the  DFT  that  leads  to  {8n}. 


To  summarize,  (5)  yields  the  cumulative  distribution  function  of  w,  and 

(6)  gives  the  impulse  areas  {<^}g  •  An  Nf-point  DFT  of  this  sequence 

(augmented  witn  zeros)  is  taken,  tnen  conjugated  and  raised  to  the  M-th 
power.  Anotner  Nf-point  OFT  is  tnen  taken  and  the  results  divided  oy  Nf. 

Tne  end  result  is  impulse  areas  [en].  The  values  returned  (by  the  second  DFT) 
for  bml+1’  •••*  bn  -1  shou^  be  zero*  as  should  all  the  imaginary  parts. 


Although  any  value  for  Nf  that  satisfies  (13)  is  permissible,  the  smallest 
power  of  2  is  most  reasonable  since  then  we  can  employ  two  FFTs  above;  this 
time-saving  feature  is  employed  here. 
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The  probability  that  decision  variable  z  equals  or  exceeds  integer  value  J 
is,  from  figure  7, 

ML 

Probability  [z  >  jj  »  en  for  0  <  J  *  ML  .  (15) 

n=J 

Tne  above  analysis  for  the  exceedance  probability  in  (15)  is  exact.  When 
signal  is  present,  (15)  is  the  detection  probability,  whereas  when  signal  is 
absent,  (15)  is  the  false  alarm  probability.  The  fundamental  input  statistics, 

P^  and  P^j  required  in  (5),  are  arbitrary. 
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QUANTIZER  BREAKPOINT  LOCATIONS 

For  a  given  number,  L+l,  of  quantizer  output  levels,  the  selection  of  the 
L  breakpoints  {bj}  should  be  done  so  as  to  optimize  the  performance  of  the 
system,  that  is,  maximum  detection  probability  for  a  specified  false  alarm 
probability.  The  equivalence  of  the  two  processors  in  figures  1  and  5  is  the 
guide  to  good  selection  of  the  breakpoints;  namely  fbj}  should  lead  to  a 
maximum  difference  in  outputs  y(m)  for  hypothesis  vs.  Hq.  However,  as 
figure  5  shows,  this  selection  of  [bj  Is  governed  by  the  probability  density 
functions  of  quantizer  input  w,  not  x;  that  is,  the  quantizer  should  take 
account  of  tne  or-ing  nonlinearity  and  the  number  of  input  channels,  N. 

A  problem  arises  here,  however,  in  that  the  probability  density  functions 
of  w  under  and  Hg  depend  on  tne  particular  value  of  input  signal-to- 
noise  ratio  that  obtains.  Thus,  the  quantizer  design  should  take  the  input 
signal-to-noise  ratio  into  account.  This  situation  is  frequently  encountered 
in  likelihood  ratio  processing,  in  which  the  optimum  processor  often  requires 
knowledge  of  absolute  levels  of  input  signal  and  noise.  Since  this  knowledge 
is  almost  always  lacking,  a  design  that  is  good  for  representative  values  of 
input  signal-to-noise  ratio,  that  is,  whicn  correspond  to  adequate  levels  of 
performance,  should  De  adopted.  If  the  input  signal-to-noise  ratio  is  larger 
tnan  tnese  representative  values,  improved  performance  will  result;  if 
smaller,  inadequate  performance  is  expected  anyway,  regardless  of  quantizer 
breakpoint  placements. 

To  get  at  tne  quantizer  design,  consider  the  probability  density  functions 
in  figure  8  for  random  variable  w  under  and  Hg.  Wg  is  a  point  beyond 
which  tnere  is  a  small  cnance  of  w  ever  reaching  under  Hg;  w^  is  a  point 
below  wnicn  w  nardly  ever  reaches  under  H^.  Generally  w^  <  Wg  for  cases 
of  practical  importance;  otherwise  near-perfect  performance  is  possible  at 
tnis  signal-to-noise  ratio.  Under  Hg,  We  would  like  to  locate  the  first 
breakpoint  b^  greater  than  Wg;  then  tne  false  alarm  probability  would  be 
substantially  zero,  regardless  of  the  remaining  [b^]  .  On  the  other  hand, 
under  H^,  We  would  like  to  locate  the  last  breakpoint  bL  less  than  w^; 
then  the  detection  probability  would  be  essentially  1. 
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These  desirable  quantizer  characteristics,  under  Hq  and  are  depicted 
in  figure  9.  It  is  Immediately  seen  that  the  desired  features  of  both  cases 
can  be  realized  for  w  <  w^  and  w  >  Wq;  that  is,  choose  q*Q  for  w  <  wj., 
and  choose  q«L  for  w  >  Wg.  However  there  is  an  inherent  conflict  in  the 
intermediate  region  <  w  <  Wg.  The  only  way  to  strike  a  reasonable 
compromise  is  to  make  q  small  near  w^  and  make  q  large  near  Wg.  That  is, 
locate  the  breakpoints  in  w^,  Wg  ,  as  indicated  in  figure  10.  We  shall 
make  tnem  equally  spaced  on  the  input  w,  that  is,  b^+1  -  independent  of^; 
nonuniform  abscissa  spacings,  such  as  in  figure  4,  are  possible  and  could  give 
sligntly  better  performance.  However,  sample  computer  runs  have  demonstrated 
that  for  L  >  4,  essentially  optimum  performance  is  attained  via  uniform 
breakpoint  spacing.  Also,  for  tne  larger  values  of  input  signal-to- 
noise  ratio,  it  will  oe  shown  that  the  optimum  processor  of  the  available 
inputs  takes  precisely  the  form  of  figure  5,  where  the  quantizer  is  replaced 
by  a  linear  gain. 

The  best  quantizer  placements  in  figure  10  obviously  depend  on  the  number, 
N,  of  input  channels  and  the  input  signal-to-noise  ratio,  since  the 
probability  density  functions  of  w  displayed  in  figure  8  depend  on  these 
quantities.  But  there  is  an  additional  less-obvious  dependence  on  M,  the 
number  of  samples  accumulated  at  the  quantizer  output.  For  larger  M,  the 
separation  of  tne  prooability  density  functions  of  z  under  Hg  and  H^  will 
become  better,  if  the  input  signal-to-noise  ratio  is  held  fixed.  But  often, 
tne  larger  values  of  M  are  employed  so  that  lower  input  signal-to-noise  ratios 
can  be  tolerated  and  yet  realize  adequate  performance  levels;  thus,  the 
probability  density  functions  of  w  in  figure  8  generally  overlap  more  for  the 
larger  values  of  M.  This  means  that  w^  will  be  smaller  and  therefore  the 
breakpoints  should  be  relocated.  Further  discussion  of  quantizer  breakpoints 
is  deferred  until  the  numerical  investigation  is  undertaken. 
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INPUT  STATISTICS 


Up  to  now,  the  statistics  of  inputs  [x^in  figures  1  and  5  have  been 
arbitrary.  We  now  specialize  to  the  case  of  Gaussian  noises;  some  other 
candidate  statistics  are  given  in  appendix  B.  In  particular,  the  cumulative 
distribution  functions  under  Hq  and  are 


P?). 


P(^(u) 


. 


(16) 


respectively,  where  $  is  the  Gaussian  cumulative  distribution  function 

x  x 


$(x)  -  j*  dt  (2»)”1/2  exp(-t 2/2)s  J  dt  <j>(t)  . 


(17) 


Tne  means  of  xn  are  itiq  and  m^  under  Hq  and  H^,  respectively,  while  the 
standard  deviation  is  the  common  value  o  in  both  cases.  For  later  use, 
we  define  the  deflection  statistic  of  the  inputs  as 


mrmo 

d1  =  a  ‘ 

Reference  to  (5)  and  (6)  indicates  that  we  need  the  quantities 


(18) 


wnere  we  nave  used  (16),  (17),  and  defined  the  normalized  breakpoints  of  the 
quantizer  as 


* 


bi"m0 


for  1  <,  J  £  L 


Then  (5)  and  (19)  yield 


(20) 
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P 

w 


[*  (s) 


for  H, 


1 


for  H 


1 


for  1  *  X  £  L 


Tnese  are  tne  quantities  needed  in  (6)  for  tne  areas  [<^}  in  figure  6. 
(If  the  noise  standard  deviations  were  oq  and  under  Hq  and  Hj, 
instead  of  tne  common  a  in  (16),  the  only  changes  would  be  to  replace  ^ 

oq  , 

in  the  second  line  of  (19)  and  (21)  by  —  (vB  -d 
next  section  are  based  upon  this  Gaussian  noise  example. 


.))• 


The  results  i 


(21) 


n  the 
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RESULTS 


By  combining  the  general  result  for  the  cumulative  distribution  function 
of  w  in  (5)  with  the  Gaussian  example  in  (16),  we  obtain 


if-?) 

-  $N-1  (v)  |  (v-d.) 

(22) 

where  we  define  normalized  variable 

U-fTl 

0 

V  ■  - 

a 

(23) 

and  have  used  (18).  Tne  probability  density  function  of  w  under  is 


»wl“>  -3?pw‘u)  "  o  3v{  $<*-"(>} 

-  j  [(N-l)  I(v-dt)*(v)  *  f  $N_Z(v)  ,  (24) 


where  we  used  (17).  This  probability  density  function  is  plotted  in  figure  11 


Figure  11.  Probability  Density  Function  of  w  for  N  =  3 


19 


TR  6815 


for  N*3  and  several  values  of  d^.  if  we  select,  for  example,  d^=5,  the 
points  Wj  and  wg  in  figure  8  correspond  approximately  to  v  -  1.4  and  4.1 
in  figure  11.  Tne  curve  for  d^*o  is  the  probability  density  function  of  w 
under  HQ. 

Tne  false  alarm  probability,  as  given  by  (15)  et  seq.,  is  plotted  in 
figure  12  for  an  example  with 

N«3  input  channels, 

M»5  time  samples  accumulated, 

L«7  quantizer  breakpoints, 

£vgj  ■  1.4(. 45)4.1,  normalized  breakpoints.*  (25) 

(Since  L+l«8  here,  this  is  called  a  3-bit  quantizer.)  Since  threshold  J  can 
only  take  on  integer  values,  the  false  alarm  probability  is  only  defined  at 
those  values,  and  is  so  indicated  by  crosses  in  figure  12.  The  detection 
probability  PQ  is  plotted  vs.  the  false  alarm  probability  Pp  ,  in  figure  13, 

by  eliminating  tne  parametric  dependence  of  botn  on  J.  Again,  both  PQ  and 
Pp  are  only  defined  at  discrete  points,  as  indicated  by  crosses;  straight 
lines  have  been  drawn  between  these  points  for  ease  of  association  of  values. 
(In  tne  curves  to  follow,  tnese  crosses  are  suppressed.)  Tne  program  for  the 
generation  of  figures  12  and  13  is  given  in  appendix  C. 

The  cnoice  of  oreakpoints  in  (25)  has  realized  near-optimum  performance 
for  L*7  and  d^5,  in  the  upper-left  corner  of  the  plot.  The  curves  for  the 
smaller  values  of  input  signal-to-noi se  ratio,  tnat  is  small  d^,  are  more 
crowded  together;  this  reflects  the  usual  small-signal  suppression  that  is 
cnaracteristic  of  nonlinear  processors. 

To  demonstrate  tne  effects  of  a  bad  choice  of  quantizer  breakpoints,  the 
previous  example  in  (25)  is  rerun,  with  normalized  breakpoints 

fa)  =  1, 1,1,1, 1,1, 2,  (26) 

instead  of  the  uniform  spacing.  Tne  results  in  figure  14  illustrate  much 
poorer  performance  than  figure  13,  in  addition  to  a  very  erratic  appearance. 


*  The  notation  a(D)c  denotes  tne  sequence  a,  a+b,  a+2b,  ....  c-b,  c. 
2U 
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Figure  13.  Operating  Characteristics  for{v^  =  1.4(. 45)4.1 


22 


.999999 


.999999 


The  curves  for  constant  must  have  a  nonnegative  slope,  and  never  cross, 
but  can  have  a  wide  variety  of  shapes.  A  similar  bad  placement  for  a  2-bit 
(L*3)  quantizer  is  given  in  figure  15.  This  jagged  behavior  of  the  operating 
characteristic  is  typical  when  the  breakpoints  are  bunched  together  instead  of 
being  uniformly  spaced. 

The  next  example  is  for 

N-3,  M»16,  L-7,  fy]  -  0(1/3)  2  ,  (27) 

and  is  displayed  in  figure  16.  This  can  be  compared  directly  with  the  results 
of  ref.  2,  figure  5.  Both  require  d^l.22  for  PF-1(T3,PD*.5,  and  both 
require  about  d^i.75  for  Pp»10~®,  P0=.5.  Thus  the  simplified  analysis 
in  ref.  2  is  very  reliable  for  large  M,  where  it  is  reasonable  to  expect 
Gaussian  statistics  to  hold. 

In  order  to  see  if  better  performance  is  attainable  by  modifying  the  break¬ 
points,  we  return  to  the  probability  density  function  in  figure  11  (for  N=3) 
and  observe  that  if  we  want  to  optimize  for  d^»3  (i.e.,  top  left  corner  of 
figure  16),  we  should  choose  w^sO,  «2*4.  however,  the  narrowing  effect 
on  the  decision-variable  probability  density  function,  due  to  the  averaging 
caused  by  large  M,  indicated  (by  trial  and  error)  that  the  best  normalized 
breakpoints  were 

M  -  ,5( .5)3.5  .  (28) 

The  corresponding  operating  characteristic  is  displayed  in  figure  17;  it  is 
slightly  cotter  than  figure  16.  Thus  the  significant  modification  in 
normalized  breakpoints  from  figure  16  to  figure  17  did  not  yield  significantly 
better  performance,  for  this  example  with  L=7. 

The  next  example  is  run  for  comparison  with  ref.  2,  figure  8.  Namely  we 

have 

N=3,  M=32,  L=7  ,  (29) 

and  three  different  quantizer  breakpoint  sets: 
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Figure  16.  Operating  Characteristics  for  M  =  16,  [vp]  =  0( 1/3)2 
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Figure  17.  Operating  Characteristics  for  M  =  16,  [v.]  =  . 5( . 5)3. 5 
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fo(l/3)2  in  figure  18" } 

\  0( .5)3  in  figure  19 

I  0( .7)4. 2  in  figure  20 


J 


(30) 


The  results  in  figures  18  through  20  are  in  almost  perfect  agreement  with  the 
simplified  analysis  in  ref.  2.  The  normalized  breakpoints  in  figure  19  are 
best  in  the  intermediate  range  of  detection  probabilities;  figure  18  is  just 
sligntly  better  for  the  hign  performance  region  of  d.  =  2. 2. 


We  now  present  a  series  of  comparisons  where  some  of  the  parameters  are 
held  constant,  while  tne  remainder  are  varied  in  order  to  determine  the  effect 
upon  the  operating  characteristics.  The  first  comparison  is  for  M=5,  N=10, 


and  tne  quantizer 

varied  as  follows: 

L*7, 

fa]  *  1.4(. 45)4.1 

in  figure  21 

1*3, 

fa}  «  2( .75)3.5 

in  figure  22 

(31) 

L-l, 

vA  *  2.75 

in  figure  23 

These  correspond  to  3-bit,  2-bit,  and  1-bit  quantizers  respectively,  where  the 
normalized  breakpoints  have  been  chosen  in  each  case  so  as  to  optimize  the 
performance  for  d-*4.  Increasing  L  beyond  7,  and  changing  the  breakpoints, 
failed  to  improve  the  operating  characteristic  noticeably  above  that  of  figure 
21.  The  increase  in  the  input  deflection,  d^,  needed  to  maintain  the  same 
performance  at  Pp=l-PgS5 10”^’^  is  approximately  4.5/4  for  L=1  vs  t=7. 

If  inputs  fxn}  are  interpreted  as  voltages,  this  corresponds  approximately 
to  a  1  dB  degradation  for  the  hard  clipper,  1*1. 


Tne  next  series  is  for  M=5  and  tne  quantizer  fixed  at  the  3-bit 
characteristic  in  the  first  line  of  (31).  N  is  varied  over  the  values  1,  5, 
10,  &,  40  in  figures  24-28,  respectively.  The  slight  improvement  in 
performance,  that  might  ensue  from  modifying  the  breakpoints  at  each  N,  was 
not  investigated  in  tnis  comparison.  Tne  effect  of  increasing  N  is  to  degrade 
the  performance,  since  the  or-ing  must  select  one  of  the  input  channels  for 
accumulation,  and  it  will  not  always  pick  the  signal-bearing  cnannel.  The 
increase  in  d.  required  to  maintain  Pp=l— Pq^s-  10“^  is  approximately 
5.25/4  =  2.35  dB  for  N=40  vs  N=l. 
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Figure  19.  Operating  Characteristics  for  M  =  32 ,  =  0(  .5)3 
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Figure  20.  Operating  Characteristics  for  M  =  32,^}  =  0( .7)4.2 
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Figure  21.  Operating  Characteristics  for  L  =  7 
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Figure  23.  Operating  Characteristics  for  L  =  1 
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Figure  25.  Operating  Characteristics  for  N  =  5 
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Figure  27.  Operating  Characteristics  for  N  =  20 
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Tne  last  series  investigates  the  effect  of  varying  M.  In  figures  29 
through  33,  M  takes  on  the  values  1,  2,  5,  10,  and  20,  respectively.  The 
effect  of  increasing  M  is  to  realize  better  performance  with  smaller  input 
signal-to-noise  ratios.  Tne  decrease  in  d.  allowed,  in  order  to  maintain 
PF-1-PD*10-3,  is  approximately  2.4/6. 8  =  -9  dB  for  M=20  vs  M*l.  When  M 
is  made  large,  the  approximate  analysis  in  ref.  2  can  be  used  with  confidence; 
tnis  is  fortunate,  since  in  the  case  of  very  large  M,  the  OFT  size,  ^ 

ML+1,  required  here  may  not  be  easily  attainable  on  some  computer  setups. 
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Figure  30.  Operating  Characteristics  for  M  =  2 
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Figure  32.  Operating  Characteristics  for  M  =  10 
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Figure  33.  Operating  Characteristics  for  M  =  20 
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DISCUSSION 


Tne  large  number  of  possible  combinations  of  values  of  N,  M,  L,  and 

precludes  an  exhaustive  compilation  of  results.  Instead  we  have  presented 
some  representative  examples  and  give  a  program  in  appendix  C  by  which  the 
user  can  investigate  his  particular  situation  and  alternatives.  This  program 
gives  exact  results  for  any  quantizer,  provided  only  that  ML  is  not  too  large 
and  that  round-off  errors  do  not  get  out  of  hand.  For  extremely  large  M,  a 
Gaussian  approximation  for  the  decision  variable  is  justified  and  the  analysis 
of  ref.  2  is  applicable. 

If  tne  quantizer  is  not  specialized  to  tne  equal  ordinate  spacings  of 
figure  3,  but  is  of  the  general  form  depicted  in  figure  2,  the  performance 
analysis  is  more  difficult.  However,  the  characteristic  function  of  the 
decision  variable  is  still  capable  of  a  closed  form  expression;  see  appendix 
D.  Evaluation  of  tne  cumulative  distribution  function  of  the  decision 
variable  is  possible  via  one  FFT,  according  to  the  methods  given  in  refs.  6 
and  7.  No  numerical  investigation  nas  been  undertaken  of  this  case. 

In  appendix  E,  the  form  of  the  optimum  processor  operating  on  N  input 
cnannels,  of  whicn  only  one  may  contain  a  signal,  is  derived  and  then 
specialized  to  the  Gaussian  input  example  of  (16).  This  optimum  processor,  in 
general,  requires  knowledge  of  the  absolute  levels  of  the  input  signal  and 
noise.  However,  for  d^  >  2,  tne  form  of  the  optimum  processor  approaches 
that  of  figures  1  or  5,  where  the  quantizer  is  replaced  by  a  linear  device, 
and  tne  absolute  level  knowledge  is  no  longer  required.  Thus  the  performance 
of  the  system  considered  here  should  be  nearly  optimum  for  large  L  and  well- 
placed  Dreaxpoints. 

Exact  analysis  of  the  system  of  figures  1  or  5,  with  a  linear  gain  instead 
of  a  quantizer,  is  more  difficult  than  that  given  here.  A  simplified  second- 
moment  analysis  was  presented  in  ref.  1;  an  exact  analysis  is  possible  and 
will  be  presented  by  the  author  in  a  future  report. 
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A  detailed  comparison  of  the  exact  performance  results  obtainable  via  this 
report,  with  the  second-moment  results  given  in  refs.  1-5,  has  revealed 
excellent  agreement  over  a  wide  range  of  parameter  values.  However,  until  an 
extensive  thorough  investigation  of  the  two  approaches  is  made  for  a  wide 
range  of  values  of  N,  M,  L,  {v^},  PF,  and  PD,  it  is  difficult  to  state 
exactly  where  the  earlier  approximate  analyses  can  be  used  with  full 
confidence.  This  time-consuming  investigation  has  not  been  undertaken; 
however,  the  program  in  appendix  C  affords  the  mechanism  whereby  this 
comparison  can  be  conducted.  The  statements  here  regarding  large  M  and 
moderate  Pp  can  only  be  made  quantitative  after  this  study  is  completed. 
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Appendix  A 

INTERRELATIONS  BETWEEN  CHARACTERISTIC  FUNCTIONS  AND 
PROBABILITY  DENSITY  FUNCTIONS  OF  DISCRETE  RANDOM  VARIABLES 


Suppose  random  variable  x  is  limited  to  the  integer  values  0,  1, 
and  that  the  probability  of  takirg  on  value  n  is  an.  That  is,  the 
probability  density  function  of  x  is 

N 

px(u)  -  ^  an  £(u-n)  . 

n«0 

The  characterisic  function  of  x  is  then 

<*  N 

.  fx(F)  -  J  du  exp(ifu)  px(u)  =  an  exp(i?n)  ; 

n*0 


this  function  has  period  2*  in  J. 


N, 


(A-l) 


(A- 2) 


Now  let  integer  M  be  selected  such  that 


M  >  N+l  .  ( A— 3 ) 

Tnen  consider  M  samples  of  characteristic  function  fx  at  increment  2n/M; 
that  is,  consider  the  set  of  samples 


fx(m2ir/M)  for  0  <  m  <  M-l  .  ( A-4 ) 

Now  let  us  take  an  M-point  OFT  of  these  samples,  and  scale  by  1/M;  that  is, 
for  0  <  k  <  M-l  , 
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M-l 

g  ^  exp(-i2wink/H)fx(m2w/M) 
m»0 

M-l  (\ 

»g  exp(-i 2wmk/M)  ^  ap  exp(i 2*nm/M) 
m*0  n-0 


N  M-l 

5  an  R  5  exp^  2i(n-k)m/M) 
n=0  m-0 


^  f(M) 

2.  an  *n-k 
n«0 


’a^  for  0  <  k  i  N 


. 


for  N  <  k  <  M-l 


(A— 5 ) 


Here,  in  the  second  line,  we  used  expression  (A-2)  for  the  characteristic 
function,  and  in  the  last  line,  we  used  (A-3). 


Expression  (A-5)  states  that  the  scaled  M-point  DFT  of  the  set  of 
cnaracteristic  function  samples,  { A— 4 ) ,  yields  precisely  the  areas 

|a^N  of  the  impulses  in  probability  density  function  px,  provided  tnat 

M  >  N+l.  Tne  values  returned  by  tne  DFT  for  aN+1,  ....  aM-1  should  all  be 
zero. 

More  generally,  if  random  variable  x  is  limited  to  the  values 

hQ,  h0+ah,  ....  hQ+N  an,  (A-6) 

the  characteristic  function  takes  the  form 

N 

fx(F)  =  exp(i?hQ)  ^  an  exp( ijn  ah)  .  ( A-7 ) 

n=0 

In  this  case,  tne  sample  set  that  must  be  subjected  to  an  M-point  DFT  is  not 
(A-4),  but  rather 


fx(lf)  ex*(-i  If  no)  for  0  S  "  *  "-1 
Wnen  scaled  Dy  1/M,  tnis  OFT  yields  areas  directly. 


( A— 8 ) 
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Appendix  B 

ALTERNATIVE  INPUT  STATISTICS 

Here  we  give  the  equations  for  two  other  typical  random  processes  that 
could  serve  as  candidates  for  inputs  to  the  nonlinear  system  of  interest.  No 
numerical  results  have  been  evaluated  for  either  of  the  following. 

CHI-SQUARED  RANDOM  VARIABLES 


Tnis  case  could  correspond  to  Gaussian  noise  with  additive  Gaussian 
signal,  after  passage  through  a  square-law  device  and  summation.  That  is, 
under  H^,  consider  tne  signal-bearing  channel  input  to  be  the  sum 

2D 


x(m)  .  ^  £k(m)  ♦  nk(m)J 


( B— 1 ) 


where  £sk(m)j  and  [nk(m)]  are  independent  zero-mean  Gaussian  random 

2  2 

variables  with  variances  o$  and  on  for  the  signal  and  noise,  respectively, 
x  is  a  Chi-squared  variate  with  2D  degrees  of  freedom. 


2  2  2 

Letting  °i  =  °s  +  °n  »  tne  probability  density  function  of  x  is  given  by 


p(i){u)  x  exp(--p-) 

(D-l)!  (2o-J  '  2o\' 


for  u  >  0 


(B-2) 


Tne  cumulative  distribution  function  of  x  is,  via  repeated  integration  by 
parts,  given  by 


P(^(u)  =  1  -  exp 


for  u  >  0 


(B-3) 


where  (ref.  8,  6.5.11) 


( B— 4 ) 
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is  tne  first  0  terms  of  exp(t). 


The  cumulative  distribution  function  of  x  under  HQ  follows  immediately 

2  2  2 
from  (B-3)  Dy  setting  os  *  0  and  identifying  ag  =  op  : 


P^(u)  =  1  -  exp 


71 

o. 


for  u  >  0 


(B-5) 


In  fact,  (B-3)  and  (B-5)  could  be  used  as  input  statistics  for  the  analysis 
contained  here  in  the  main  body  of  the  report,  without  the  need  for 
interpretation  (8-1);  (B-l)  merely  lends  the  physical  interpretation  of  x  as 
the  sum  of  a  number  of  diversity  inputs. 

According  to  (5)  and  (6),  we  need  the  quantities 


p(x}(A)  - 1  -  exph)  eD-i  (*) 

AD 


p(i)  M  ’1-exp(-^vx]  eo-i  R  * 


for  1  <  X  i  L  ,  ( B-6) 


J 


where  now  tne  normalized  breakpoints  are  defined  as 

bp 

v  «  — for  1  <J  <  L 

1  2of 


Of  course,  we  should  always  select  b^  >  0,  since  x  is  never  negative. 


(8-7) 


NONCENTRAL  CHI-SQUARED  RANDOM  VARIABLES 


This  case  corresponds  to  Gaussian  noise  with  additive  deterministic 
signal,  after  passage  througn  a  squarer  and  summation.  Under  let  the 
input  be  composed  as  follows: 


x(m) 


( B— 8 ) 


where  [cK(mjij  are  constants,  and  noises  [i\(in)]  are  independent  zero-mean 

2 

Gaussian  random  variables  with  variance  on>  For  example,  x(m)  corresponds  to 
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the  sum  of  D  squared-envelopes  of  the  output  of  a  narrowband  filter  subjected 
to  a  sinewave  and  Gaussian  noise.  Tne  probability  density  function  of  x  in 
( B— 8)  is 

.0) 


p  x  ' <“> 


1  faV  m  “  \  I  /aVtr\ 

r-2  (aST)  exp(-  T  - 

°n  V  V  2V  \  "  / 


for  u  >  0, 


where 


..i- 

°n 


20  -ll/2 

]>  ck<m> 

Lk«l 

is  a  measure  of  the  total  signal-to-noise  ratio  of  random  variable  x(m). 

(a  is  generally  a  function  of  m.)  The  cumulative  distribution  function  is 

,(1) 


( B— 9 ) 


( B— 10 ) 


P'*'(u)  -  1  -  Qq 


(•■f) 


for  u  >  0 


( B— 1 1 ) 


where  the  Q-function  is  (ref.  9) 


Q0(a,b)  r  j* dt  t(|)  exp^- 


0-1  (  t2  ♦  a2> 

exp(-  *  2  J  Tj-i  (at) 


(B-12) 


Tne  probability  density  function  and  cumulative  distribution  function 
under  riQ  follow  from  (B-9)  and  (8-11)  by  setting  a=*0,  where  we  presume  that 
{cl<(mj}  represent  the  signal  components  in  (B-8);  there  results 


P^(u) 


.0-1 


(D-D!  (2»n2) 


u  exp(-  -h 

i  v  2°„ , 


for  u  >  0 


( B— 1 3 ) 


and 


u 


p<>)-l-exp(-^je0w^ 


for  u  >  0 


( B— 14 ) 


According  to  (5)  and  (6),  we  need 

plx’W  =  1  -  exp(-vi)  eD-l(vj) 

P<x'W  *  1  -  %k' 


for  1  <  X  £  L 


( B— 15) 


where  normalized  breakpoints 

b 


^or  1  <.  /(  <  L 

1  2a. 


( B— 1 6 ) 


n 


Again  >  0  since  x  in  (6-8)  can  never  be  negative. 
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Appendix  C 


PROGRAM  FOR  DETECTION  AND  FALSE  ALARM  PROBABILITIES 


10 

20 

30 
40 
50 
60 
70 
80 
80 
100 
110 
120 
130 
140 
150 
160 
170 
ISO 
180 
200 
210 
220 
230 
240 
250 
260 
270 
280 
280 

3  0  O 

3 1  O 
320 
330 
340 
350 
360 
870 
380 
380 
400 

4  10 
420 
4  30 
440 
4  5  0 
4  6  0 
470 
480 

4  8  0 

5  0  0 


!  DETECTION  CHARACTERISTICS  FOR  QUANTIZERS,  GREATEST -OF ,  AND  ACCUMULATOR 
!  QUANTIZER  OUTPUT  (ORDINATE)  LEVELS  ARE  SET  AT  0tl,...,L;  L+l  LEVELS 
1  QUANTIZER  ABSCISSA  BREAKPOINTS  ARE  ARBITRARY;  L  BREAKPOINTS 
M=5  !  NUMBER  OF  TIME  SAMPLES  ACCUMULATED:  M>=1 

N-3  !  NUMBER  OF  INPUT  CHANNELS  SUBJECT  10  GR-IMG:  N>=1 

L*7  !  NUMBER  OF  NON-ZERO  QUANTIZER  OUTPUT  LEVELS:  L  =1 

'  NORMALIZED  ABSCISSA  BREAKPOINTS  OF  QUANTIZER  L  NUMBERS): 

BAT A  1.4, 1.85,2.3,2.75,3.2,  3.65,4.  1 

Dirv.ix«4.5  !  MAXIMUM  VALUE  OF  Di  OF  INTEREST 

D i  step* .  5  !  INCF:EMENTS  IN  Di  OF  INTEREST 

Np  =  6  !  SMALLEST  PROBABILITY  OF  INTEREST  IS  10  -Np 

DIM  X<  1 :  1024),  Y<1  : 1024),  Axisd:  19) 

dim  vu:  1 0 > , s < l :  i0),Pwd:  ie>,Pfi.<i:  1024> 
redim  v<  l : L),S(i:L>,Pw< l ; u>, Pf  *< i : m*l  .■ 

READ  V ( * > 

PRINTER  IS  0 

PRINT  "NUMBER  OF  TIME  SAMPLES  ACCUMULATED:  M  - " ; M 

PRINT  "NUMBER  OF  INPUT  CHANNELS  SUBJECT  TO  Of-IHG:  N  *';N 

PRINT  "NUMBER  OF  NON-ZERO  QUANTIZER  OO'-'.T  lE  £l;!  l.  *  ;L 

PRINT  "  QUANTIZER  OUTPUT  (ORDINATE  LEVELS  APE  SET  AT  0,1 . L." 

PRINT  "  QUANTIZER  NORMALIZED  'ABSCISSA)  E'F:£At: PO I NTS  ARE  AT:" 

FOP  J*1  TO  L 
PRINT  V(J); 

NEXT  J 
PRINT 

PRINT  "INPUT  VOLTAGE-SNP  Di  VARIES  FROM  0  TO  ' ; D l  ft.  a  ;  "  i  N  STEPS  OF":In=t 

PRINT 

PRINT  "THE  GRAPH  BELOW  GIVES  T.hE  '.FALSE  ALARM'  s'e  OBA£  I L I TY  ThAT  THE" 
PRINT  "SYSTEM  OUTPUT  IS  GREATER  ThAN  OF  EQUAL  TO  \  FOP  J  =  1  TO  ML.' 
PRINT  "THE  INPUT  SIGHAL-TO-NOISE  RATIO  IS  ZERO  THIS  GRAPH .  " 

M  0  *  M  *  L 
M  1  =M0+ 1 
M2*M  2 
N 1  =  N  -  1 

FOP  J=2  TO  10 
Nf  =  2  J 

IF  Nf  >M0  THEN  420 

NEXT  J 

PRINT 

PRINT  "ARRAYS  X,Y,Pf*  APE  NOT  D I  MEN  SI  ONE  I  !  ►•F  IE  ENOUGH" 

STOP 

PEDIM  X-  1  :  Nf  :• ,  Y <:.  1  :  Nf  • 

FOP  J*1  TO  L 
S(J)=FNPm  ..  V  <  J  '  HI 
next  j 

F 0 P  D  l  =  O  TO  D  i  r.i  a  STEP  D  l  a  t  e  p 
FOR  J=1  TO  L 

P  w  '’  J  >  =  S  1  J  *  +  FNPh  i  ■  \!<  J  •  -D  l  >  1  E  . 1  ■.  A  "  I  :  AND  21 

NE  T  J 
MAT  =ZEP 
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510  MAT  Y*ZER 

520  XM)*Pu',l)  !  EQUATION  6;  Alpha-:  T,.  in  X  ■  J  + 1  > 

530  FOR  J«2  TO  L 

540  X( J)*Pw< J)-Pw< J-l> 

550  NEXT  J 

560  X<L+l>*l-Pw<L> 

570  CALL  Ff t  <Nf ,X<*),Y<*>) 

580  FOR  J*1  TO  Nf  !  <X-i Y>  'M 

590  T=<X(J>  '2+Y<J>A2;~M2 

600  A=»M*FNArg<X<  J) ,  Y<  J) ) 

610  X<  J)*T*COS<A'- 

620  Y<J)*-T*SIN<A> 

630  NEXT  J 

640  CALL  Fft <Nf , X<*> , Y<») > 

650  FOR  J*1  TO  Ml 

660  X<J;*X(J)/Nf  1  BETA  OF  FIGURE 

670  NEXT  J 

680  IF  Di>0  THEN  1220 

690  PLOTTER  IS  "GRAPHICS" 

700  GRAPHICS 

710  Np 1 *Np* 1 

720  Np2*Np*2+l 

730  Axii<Npl)*0 

740  FOR  J«1  TO  Np 

750  T*FNInuphi < . 1AJ> 

760  Ax i 2  <  Np 1 - J ) =T 

770  Ax i 3- Npl ♦ J >*-T 

780  NEXT  J 

790  SCALE  0,  M0,  Axi  s<  1  >  ,  Axi  i  <Np2'- 

800  FOR  J  =  0  TO  M0  STE!P  5 

310  MOVE  J , Ax i 2  <Np2> 

820  DRAW  J,Axis<l) 

330  NEXT  J 

340  FOR  J*1  TO  Np2 

850  MOVE  0 , Ax i  i <  J  > 

360  DRAW  M0,Axii(J> 

870  NEXT  J 

880  PENUP 

3  3  0  T  —  o 

90O  FOR  J  =  M 1  TO  2  STEP  -1 

910  T  =  T  +  X  <  J  > 

920  I  =  J- 1 

930  IF  ' T  IE-1 1  >  AND  (T  1-lE-ll  -  THEN  960 

940  Pf  v  I  >  =  1 0O 

950  GOTO  1O50 

960  A*FN  I  rwph  i  <.  T  j- 

97  0  Pt'a-.  I)*H 

980  PLOT  I, A 

990  1  GOTO  1O50  1  TO  ELIMINATE  CFGSSES.  I'  SECT  T  -  .  :  ’  -  “  F  1C  T 

10O0  MOVE  I,A-h  i s - Np7  1  * . 0O8 
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1010  DRAW  I , A+Axi »<Np2>*. 008 

1020  MOVE  I - . 004*M0 , A 

1030  DRAM  I + . 004*M0 , A 

1040  PLOT  I, A 

1050  NEXT  J 

1060  PENUP 

1070  DUMP  GRAPHICS 

1080  PRINT  LIN< 1 ) 

1090  PRINT  "THE  GRAPH  BELOM  IS  A  PLOT  OF  DETECTION  PROBABILITY  VERSUS" 
1100  PRINT  "FALSE  ALAF'M  PROBABILITY,  FOR  THE  VALUES  OF  Di  GIVEN  ABOVE." 
1110  GCLEAR 

1120  SCALE  Axi  s<  1 )  ,  Axi  *<  Np2)  ,  Ax  i  s  <  1  > ,  Axi  * <  r<o2  > 

1130  FOR  J*1  TO  Np2 
1140  MOVE  Ax i  *<  l  )  ,  Axi  i. <  J> 

1150  DRAW  Axis<Np2>,  A>:  i  *  C  J) 

1160  NEXT  J 

1170  FOR  J»1  TO  Np2 

1180  MOVE  Ax i 4< J ) , Ax i i v Np2 ) 

1190  DRAW  Axi 4<  J) , Ax i s  < 1 > 

1200  NEXT  J 
1210  PENUP 
1220  T-0 

1230  FOR  J*M1  TO  2  STEP  -1 
1240  T*T*XC-J) 

1250  IF  < T< 1 E- 1 1 >  OR  <  T M - 1 E - 1 1  ■  THEN  1360 
1260  B*Pf  »<  J- 1 ) 

1270  IF  ABS<  B ) >7  THEN  1360 
1280  A*FNinvphi <T> 

1290  PLOT  B ,  A 

1300  !  GOTO  1360  !  TO  ELIMINATE  CROSSES,  INSERT  ihIS  INSTRUCTION 

1310  MOVE  B,  A-Axi  008 

1320  DRAW  B, A+Axi i <Np2>*. 008 

1330  MOVE  B-Axi s<Np2)* . 003, A 

1340  DRAW  B+AxiiCNp2>»,008,  A 

1350  PLOT  B ,  A 

1360  NEXT  J 

1370  PENUP 

1380  NEXT  Di 

1390  DUMP  GRAPHICS 

140O  PRINT  LINC6) 

1410  PRINTER  IS  16 
1420  END 
14  30  1 

1440  DEF  FNAr  g  <  X ,  Y )  !  PRINCIPAL  -PGVMtNT  Or  -nr' 

1450  IF  X»0  THEN  A  = .  5 "  P I  ♦  S G N <  Y  ’■ 

1460  IF  X  O  THEN  A*ATH< Y  X) 

1470  IF  X  \  0  THEN  A  =  A  +  F'  I  *  1  -2*  <  r'  O  1  > 

1430  RETURN  A 
1490  FNEND 
'  500  1 
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1510  DEF  FNPhi(X)  !  CUMULATIVE  GAUSS  I  AH  D I STP I  PUT l OH 

1520  IF  ABS<X>  >5.14  THEN  1780 
1530  A».282842712475*X 

1540  C*COS<A> 

1550  S«SIN<A> 

1560  B*2*C 

1570  A«B*C-1 

1530  C*A*<  1 . 253675 IE-1 8+B+7.  10005E-20+A+7. 4517E-21  ■ 

1590  C»A*<1 .533423425E -16+B*1 .  0 1 649277E- 1  7+0 

1600  C*A*  < 1 • 36760444757E-14+B*1 • 0601364636E- 15  +  C > 

1610  C*A*<8.89786526722E-13+B*8. 060683 38945E-14+0 

1620  C*A*'!4. 2261614431  8E- 11  +B*4. 46968229249E-12+C  ' 

1630  C*A*  ■  1 . 4666061 4234E-9+B*l . 808435378 10E- 1 0  +  0 
1640  C»A*<3. 722523493 69E-8+B*5. 342750276 8 3E-9+C ■ 

1650  C*A*<6.91927520325E-7+B#l. 1 533099O944E- 7+C > 

1660  4 3281 169838E -6+B*l. 82 066316 364E-6+C • 

1670  C*A* <9. 4490926381  0E-5  +  B*2.  1 0404533073E-5  +  C 
1680  C*A*C6.97183792403E-4+B*l .7822S016255E-4+C > 

1690  C«A*<3.80150767935E-3+B*l . 10860645342E-3+C > 

1700  C»fi+<.0153985726157+B*.0050790696l220+C - 

1710  C * A* <. 0467755234325+ B*.  8 1 72439625887+0 
1720  C«A*< .  108630245023+B*.  0439773331941+0 

1730  C*A*< .201 339747265  +  B*.  0869S94549959+O 

1740  C*A*< . 33050152191 7  +  B*.  1 44227226362+0 

1750  C«.  703225002744+I:».24725516S140  +  C 

1760  Phi *. 5+. 0450 1531 58079*X+. 5*S*C 
1770  GOTO  1920 
1730  IF  107  THEN  1910 

1790  N  =  MAX<6,  INT<69 'AI:S<X>  >,  INT'  525  :02  1 

1800  A*  1 

1810  S*1 

1820  BM^X 

1830  C«B*B 

1340  FOR  J»1  TO  N 

1350  A*  <  1 -2+  J ) *A*C 

i860  S«S+A 

1370  NEXT  J 

13S0  Phi  *.  39394223040 J  *EXP<  -  .  5+X*X.* +ABS  B 

1390  IF  X>0  THEN  Phi »1 -Phi 

1900  GOTO  1920 

1910  Ph i =  1 

1920  RETURN  Phi 

1930  FNEND 

1 940  1 
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1950 

I960 

1970 

1980 

1990 

2000 

2010 

2020 

2030 

2840 

2050 

2O60 

2070 

2080 

2090 

2100 

2110 

2120 

2130 

2140 

2150 

2160 

2170 

2180 

2190 

2  cl  0  0 
2210 
2220 
2230 
2240 
2250 
2260 
2270 
2280 
2290 
2300 
2310 
2320 
2330 
2340 
2350 
2360 
2370 
2280 
2390 
2  4  0  0 
2  4  10 


DEF  FNIrtvphi  <Z>  !  INVERSE  CUMULATIVE  GAUSSIAN  DISTRIBUTION 

X  =  2  *  Z  - 1 

DIM  T(0: 20) , A<0: 20 > 

DATA  .992885376619, . 1 204675 1 6 1 43 , .0168781993421, .00268670443716 
DATA  . 4996347302*  E-3,  .  988982 1  86E-4 , . 2039181276E-4,  .  4 32727 1  62E-5 
DATA  . 93808 1 4 1 E-6 , . 20673472E-6 , .461597E-7, . 1 04 i 663E-7 , .23715E-8 
DATA  . 54393E-9 , . 12555E-9,  .  2914E-10, .  679E-1 1 , . 159E- 1 1 ,  . 37E-12 
DATA  . 91215880341 8, -.0162662813677,  . 4 335564  7295E-3 .  .  2 1 4  43857007E- 
DATA  . 262373 108E- 3, -.3021 09 103E-5,-. 1240686E-'’,  . 624066 IE-7 , - . 340 1 
DATA  -. 142321E-8, .3438E-10,  .  3338E-10,-.  1  46E- 1 1 , - . 8 1 £- 1 2 ,  .  3E- 1 3 ,  .2 
DATA  . 936679709020, -.0231070043091 , - . 0843742 260? '5 1 ,  -  .  57650342263 
DATA  -. 1096102231 E-4, . 23 1 0834702E-4 , . 105623360?E -4,  .  27344 123  3E-3 
DATA  . 43248450E-6.-.2033034E-7, -. 43891 54E-7, -. 1 76340 1 E-7 ,  - . 399 129E-3 
DATA  -. 1 3693E-9, .27292E-9,  .  13282E-9, . 3133E-10, . 167E-1  1  ,  -.  284E-1 1 
DATA  -.965E-12,-.22E-12 
B*ABS  <  X  ) 

IF  ABS^X)  >*.  8  THEN  B»$QR<-L0GU1-X>*<  1*X>  •  • 

IF  ABS<X)< . 8  THEM  2220 
IF  ABS-'XX  .  9973  THEN  2180 
Nrn*x*20 
RESTORE  2030 

Y*-. 339437631330 *B  +  2. 28791571626 
GOTO  2238 
Nnri*x*  1  5 
RESTORE  2O20 

Y«- 1. 54381 304237* B+2. 5654901 23 15 

GOTO  2250 

Nffiix*  1  8 

RESTORE  1980 

Y*X*X*3 . 123-1 

PEDIM  A<  0:  Nniax  > 

READ  A'.>'» 

Y2*Y*2 
T »  0  <  *  1 
T  i  1  ■  =Y 

FOR  N*2  TO  Nrnix 
TOO-Y2*T<N-l  >-T«  N-2> 

NEXT  N 
R*U 

FOR  N«M»i*x  TO  O  STEF'  -1 
P*R*A(.N."*T •  N  • 

NEXT  N 

I  iv,' phi  =SGNOO*BX>1 . 4  1421  356237 

P  E  T  U  R  N  I  ti  ‘j  p  h  i 

FNEND 

i 

SUE  Ff  t  -  N,  V<  *■  J  I  FF  T  S  uBF'C  •.  T  1 1,£  HEFE 
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Appendix  0 


ANALYSIS  FOR  GENERAL  QUANTIZER 


Tne  characteristic  function  of  decision  variable  z  is  still  given  by  (8) 
in  terms  of  f  the  cnaracteristic  function  of  y.  But  now,  from  figures  5 
and  2, 


fyif)  - 

-  exp(ijh 


exp(ijFy)  -  expirfqjwf )  -  J du  pw(u)  exp( if q Ju> ) 
,)  J  du  pw(u)  ♦  ...  +  exp(i?hL)  f  du  pw(u) 

—  oo  i" 


L-l 

-  exp(iTho)  Pw(b1)  ♦  ^exp(if^)  [pw(b,+1)  -  Pw(b,)] 

i«l 

♦  exp(iJhL)  [l  -  Pw(bL)]  ,  (0-1) 

wnere  P  is  the  cumulative  distribution  function  of  random  variable  w,  as 

W 

given  by  (5).  Tne  inputs  for  tnis  calculation  of  cnaracteristic  function  f 
are  ri,  N,  L,  fhj]  *"  and  ^}L  «  Again,  input  cumulative  distribution  functions 

and  P^  in  (5)  are  arbitrary. 

One  problem  with  tnis  quantizer  is  tnat  PQ  and  Pp  are  stepwise  functions 
of  the  decision  threshold  (which  need  not  be  integer  now)  at  irregular  points. 

A  large  DFT  size  would  be  necessary  to  track  this  behavior.  However,  operating 
cnaracteristic  plots  of  Pq  vs.  Pp  would  be  smoother  functions. 
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Appendix  E 

DERIVATION  OF  OPTIMUM  PROCESSOR 


We  allow  tne  signal,  if  present,  to  jump  randomly  and  independently 
between  input  channels  on  each  time  sample.  Let 


Y (m)  -  jxn(m)|  for  1  <  m  i  M 


(E-l) 


Under  H^,  the  probability  density  function  of  N-vector  Y(m)  is 

P^(Y(m))  “  |  p(0^(x2(m))  ...  p^(xN(m)) 

+  p^^x^m))  p^(x2(m))  ...  p^°^xN(m)j 


for  1  <  m  i  M 


Tnerefore  the  joint  probability  density  function  is 


p(1)(Y(l)  ...  Y ( M ) ) 


m*l  n»l 


(E-2) 


(E-3) 


Tne  likelinood  ratio  follows  immediately  as  the  last  product  in  (E-3);  tne 
log-likelihood  ratio  is  therefore 


^n  likelihood  ratio 


(E-4) 


Tnis  result  holds  for  arbitrary  inputs  with  probability  density  functions 
p(^)  and  p( U  . 

When  we  employ  tne  Gaussian  example  in  (16),  (E-4)  simplifies  to 
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where  data-independent  scale  factors  have  been  absorbed  in  the  threshold. 
Exact  analysis  of  ( E  —5)  is  conceivable,  but  is  exceedingly  tedious.  Also 
dj  and  a  must  be  known  in  order  to  realize  (E-5). 


For  di  >2,  it  may  be  shown,  to  a  good  approximation,  that 


Substitution  in  (E-5)  yields  the  approximate  likelihood  ratio  test 


(E-6) 


M 

^  max  {xn(mj}  *  threshold  .  ( E-7 j 

m=l  n 

Knowledge  of  d^  and  a  is  now  not  required.  Processor  ( E— 7 ;  is  just  figure 
1  or  5,  with  the  quantizer  replaced  by  a  linear  gain.  This  special  case  of 
(E-5)  is  very  important,  because  decent  performance  can  be  obtained  for 

dj  >  2,  and  this  is  exactly  where  (E-7)  is  virtually  optimum.  A  good 
approximation  to  the  performance  of  (E-7)  is  afforded  by  the  results  contained 
herein,  if  L  is  chosen  large  and  the  breakpoints  are  well-placed.  Some 
approximate  results  via  second-moment  approaches  are  given  in  refs.  1,  10,  11, 
12. 
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On  Resonance  Extraction 
And  Waveform  Fitting 
For  Transient  Data; 
Prony’s  Method 


A.  H.  Nuttall 

ABSTRACT 

This  report  explains  the  basic  philosophy  and  mathematics  of 
waveform  fitting  with  complex  exponentials,  when  the  available  data 
points  are  more  than  the  number  needed  for  a  perfect  fit.  The  con¬ 
nection  with  Prony's  method  is  developed,  some  recent  new  work  by 
Auton  and  Van  Blarlcum  Is  summarized,  and  an  eigenvector 
generalization  of  linear  prediction  is  presented.  Effort  is  still  con¬ 
tinuing  in  this  important  field  of  resonance  estimation  and  extraction, 
and  answers  to  some  Important  questions  on  sensitivity,  sampling 
rate,  and  bias  are  still  not  available. 
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ON  RESONANCE  EXTRACTION  AND  WAVEFORM  FITTING 
FOR  TRANSIENT  DATA;  PRONY'S  METHOD 


INTRODUCTION 

The  estimation  of  the  resonances  (natural  frequencies)  of  a  system,  from 
observation  of  a  noisy  response.  Is  an  important  problem  of  frequent  occurrence 
In  practical  situations.  Usually,  the  number  of  observations  is  considerably 
greater  than  the  number  of  resonances,  and  the  task  of  utilizing  these  "extra" 
data  to  reduce  the  errors  of  estimation  must  be  accomplished  without  an  exces¬ 
sive  amount  of  computational  effort  or  trlal-and-error.  Accordingly,  the 
original  exact-fit  procedure  by  Prony  has  to  be  generalized  to  a  least-squares 
approach.  In  this  manner,  the  amount  of  data  processing  is  minimized,  with 
all  the  nonlinear  processing  being  concentrated  in  the  solution  for  the  roots 
of  a  polynomial. 

The  purpose  of  this  report  is  to  develop  and  explain  this  least-squares 
solution  and  to  show  its  close  connection  to  linear  prediction.  The  first 
section,  on  Mathematical  Details,  sets  up  the  problem  definition  and  intro¬ 
duces  the  terms  necessary  to  interpret  recent  work  by  Auton  and  Van  Blaricum  [1] 
described  in  the  next  section.  Some  important  points  about  the  waveform¬ 
fitting  technique  are  explained,  and  some  possible  alternative  approaches  are 
mentioned.  A  more  general  model  is  considered  in  appendix  A,  and  a  generaliza¬ 
tion  to  linear  prediction  is  developed  in  appendix  B,  which  subsumes  forward 
prediction,  backward  prediction,  and  a  weighted  linear  combination  in  general. 


\ 


1 


TR  6639 


MATHEMATICAL  DETAILS 


IDEAL  EXPONENTIAL  MODEL 


Suppose  a  sequence  {g 


}N->. 

nrO 


of  N  points  is  given  exactly  by  the  model* 


g 


m 


I  C.  exp(a.  m)  s  I  C.  p.  for  0  £  :n  £  N-l. 
k=l  K  *  k=l  K 


(1) 


That  is,  sequence  {gmlo"1  1S  a  sum  n  complex  exponentials.  Without  loss 
of  generality,  we  presume  that  all  the  {C|<}  are  nonzero  for  1  <  k  <  n. 

Consider  the  error  (in  linear  prediction)  of  attempting  to  represent  gm 
in  terms  of  its  past  n  values;  that  is,  for  n  <  m  <  N-l,  consider  linear 
prediction  error  (where  a  =  -1) 


g  - 


M  II  I*  If  Z 

^  9m_  i  =  —  2  o  •  g  .  =  2  o-  2  C.  p. 

j=l  J  ™  J  j=0  J  m  J  j=0  J  k=l  k  k 


=  k~l  Cr  Mk  0  Z0  (  aj  Mk  ^  =  k^l  C|c  Mk  0  ^k  "  1‘**‘”an-l  Mk'an^ 

(2) 

where  we  substituted  (1)  and  interchanged  summations.  Now  we  choose  the  n 
linear  coefficients  {a . }!?  such  that 

J  • 


n  n-l 
Mk  •  “^k 


Vl^k  '  “n  =  0  for  1  s  k  =  n- 


(3) 


This  requires  solution  of  n  linear  equations  for  the  n  unknowns 
presuming  that  the  n  quantities  (u.  }!?  are  known.  In  fact,  the  general 
solution  is 


a,  =  (-1)J_1  {sum  of  all  possible  products  of  j  different  u's} 

J 

for  1  <  j  <  n; 


(4a) 


*This  can  be  generalized  to  include  terms  like  Cu 


m 


+  Dmu 


rr 


see  appendix  A. 
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that  Is, 


or 


<»!  =pi  ♦  y2  +...+  pn 


<*2  =-(MiM2+PiM3+*  •  •+MiMn+P2M3+P2M4+-  •  *+ViMn) 
•  ^  - 

an  =  (-1)°"  HiM2---Mn- 

With  this  choice  of  {c^}",  (2)  and  (3)  yield 

9m  '  °j  Vj  =  0  for  "  *  m  «  "-I. 

n 


g 


m 


1  ai  9 n_ •*  for  n  S  m  £  N-l, 
j=l  J  ,n  J 


(4b) 


(5a) 

(5b) 


N-l 

That  Is,  when  sequence  {gm)g  Is  generated  as  a  sum  of  n  complex  exponentials 
according  to  (1),  the  sequence  value  gm  can  be  determined  exactly  as  a  forward 
linear  combination  of  the  previous  n  values,  provided  that  n  <  m  <  N-l.  The 
restriction  of  m  to  this  range  is  due  to  the  fact  that  gm  is  presumed  unknown 
for  m  <  0  and  for  m  >  N-l;  thus  only  the  "valid,"  or  available,  data  are 
employed  In  (2)  and  (5b). 

It  Is  Important  to  observe  that  the  n  linear  predictive  coefficients  {a.}? 
In  (4b)  depend  on  {  but  are  completely  independent  of  the  values  of  the  J 

exponential  strengths,  or  "residues,"  {Ck>j  In  (1).  Also,  if  the  (ajlJ  were 

known  instead  of  the  then  (3)  can  be  solved  for  the  (u- }?  as  the  n  roots 

of  an  n-th  order  polynomial.  1 

A  more  general  approach  to  linear  prediction  is  developed  in  appendix  B. 

It  subsumes  the  forward  prediction  (given  above),  backward  prediction,  and  a 
weighted  linear  combination  in  general. 


ACTUAL  MEASURED  DATA 

N-l 

Nov  suppose  that  some  arbitrary  data  sequence  { fm>o  has  been  measured 
or  is  available,  and  we  want  to  choose  the  2n  parameters  in  the  exponential 

N-l 

model  (1)  such  that  the  error  of  representing  data  {fff)  by  this  model  is 

minimized  in  some  sense.  Guided  by  (5b),  we  first  let  linearly  predicted 
value 
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f  £  Z  a.  f  .  for  n  2  m  2  N-l,  (g) 

m  j=l  J  m’J 

where  the  linear  coefficients  (a-}?  are  to  be  selected.  In  particular,  we 

J  • 

define  the  prediction  error  sequence  (called  the  equation  error  in  [1]) 


n 

e  =  f  -  f  *  f  -  Z  a.  f  . 
m  n  m  m  Jsl  J  n  j 


for 


This  is  also  called  Prony's  difference  equation, 
squared  prediction  error  as* 


n  2  m  2  N-l.  (7) 
We  then  define  the  total 


E  £ 


N-l 
Z  w 
m=n 


m 


N-l 

Z 

m=n 


n 

Z  “i 
j=l  J 


(8) 


where  1  are  a  set  of  N-n  positive  weights.  E  is  called  the  quadratic 

error  In  [1] . 

Minimization  of  total  squared  prediction  error  E  by  choice  of  coefficients 
is  accomplished  by  setting 


for  1  $  k  S  n. 


(9) 


This  results  in  n  linear  equations  in  the  n  unknowns  We  solve  these 

equations  for  the  that  minimize  prediction  error  £. 

We  must  point  out  an  alternative  approach  to  the  minimization  of  E. 

One  could  instead  minimize  the  Chebyshev  error;  that  is,  we  could  choose  the 
{a.}!?  in  (7)  so  as  to  minimize  the  quantity 

J  ^ 


max 

nSmSN-1 


f 


m 


n 


Z 

j=l 


a 


m- 


j 


(10) 


That  is,  the  maximum  error  in  prediction  is  minimized.  Although  this  approach 
yields  nonlinear  equations  in  the  {a.}!?,  efficient  linear  programming  techni- 

ques  exist  for  this  problem.  How  well  this  minimax  error  criterion  compares 
with  the  total  squared  error  criterion  is  not  known, 


*We  are  presuming  real  data  sequences  here;  generalization  to  complex  data  is 
possible. 
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Given  the  values  for  (ak}!|\  whether  obtained  via  (9)  or  (10),  we  can  now 
solve  (3)  for  the  f uk ^ 1 •  $ome  these  latter  values  may  be  complex,  even 
though  all  the  {ak}J  are  real  for  real  data  (f^"1;  this  situation  is 
treated  in  [2],  p.  380. 

Guided  now  by  (1),  we  next  let  model  data  value* 


fm  s  ^  Ck  for  0  S  m  S  N-l. 

Then  we  define  data  error  sequence  (called  the  true  error  in  [1]) 


*.  £  ^  Ck  for  0  S  .  S  N-l. 


In  a  similar  fashion  to  (8),  we  also  define  the  total  squared  data  error  as 


E  =  1  =  1 

m=0  m  m  m=0 


w  (f  *  1 
m  V  m  k=1 


where  (w  „},!  are  a  set  of  N  positive  weights.  To  minimize  total  error  E, 

we  set  m  0 


—  =  0  for  IS  j  S  n, 
3C . 

J 


thereby  obtaining  n  linear  equations  in  the  n  unknowis  {C.}?.  (The  quanti ties 
(uk)j  are  already  known  at  this  point;  see  the  discussion  preceding  (11)). 

We  solve  these  n  equations  for  the  {C.}?  that  minimize  E. 

w  ^ 

An  alternative  approach  to  the  minimization  of  E  is  to  minimize  the 
Chebyshev  error;  that  is,  choose  the  {C,  )!?  in  (12)  so  as  to  minimize  the 
quantity  1 

n 

max  f-  "  Z  C.  u™  .  MRl 

OSmSN-l  *  k=l  k  k  U5) 


*This  presumes  that  all  the  roots  { u ^ ^  are  distinct;  if  on  the  other  hand. 

we  had,  for  example,  =  Uj,  then  we  need  +  C2mpj  rather  than 
^  m  ,  *  fii 

Cl“l  C2"2- 
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Again,  the  performance  quality  of  (10)  and  (15)  is  not  known. 

At  this  point,  we  have  a  "fitted"  waveform, 

n 

I  C.  pV,  for  0  2  m  2  N-l,  (16) 

k=l  K  * 


N-l 

to  the  original  given  data  sequence  { fm>0  .  However,  it  should  be  observed 

that  the  fit  was  obtained  via  a  two-staqe  sequential  procedure.  Namely,  we 
first  minimized  total  prediction  error  §  to  find  the  linear  coefficients 

and  from  them,  solved  the  polynomial  of  (3)  for  its  roots  ^ ^ • 

(These  latter  quantities  are  called  the  resonances  in  [1]).  Then,  with 

these  known  values  for  {uk}",  tota^  data  error  E  was  minimized,  thereby 

determining  the  strengths  (residues)  {C^}"  of  each  of  the  known  exponential 

components 

Both  error  definitions,  (7)-(8)  and  (12)-(13),  utilize  and  "fit"  the 

N-l 

available  data  sequence  {f  }Q  ,  but  in  two  different  senses,  the  first  via 
linear  prediction,  and  themsficond  via  an  exponential  model.  The  worst  non¬ 
linear  data  processing  encountered  in  this  two-stage  procedure  is  the 
solution  of  an  n-th  order  polynomial,  (3),  for  all  its  roots  -  This 

sequential  procedure  will  not  realize  as  small  an  error  as  direct  minimiza¬ 
tion  of 


,/ 


n 

z 

k=l 


(17) 


via  simultaneous  choice  of  { ^  and  (u^.  However,  this  latter  approach 

is  highly  nonlinear  in  the  and  no  direct  (nonrecursive)  solution  is 

known.  Of  course,  a  gradient  search  on  (17)  could  be  employed,  using  as 
starting  values,  those  obtained  above  via  the  two-stage  sequential  procedure. 
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SOME  RECENT  WORK 

The  source  of  the  following  results  and  comments  is  the  work  by  Auton 
and  Van  Blaricum  [1].  The  solution  for  the  coefficients  (cijljj1  in  (9)  is 

called  the  reduced  or  inhomogeneous  solution;  see  [1],  vol .  I,  p.  2-5. 

This  traditional  solution,  unfortunately .  tends  to  zero  as  the  white 
(independent)  noise  component  in  {f^ljj-1  gets  larger.  A  remedy  to  this 

undesired  behavior  is  furnished  by  employing  instead,  the  weakest  eigen¬ 
vector  of  the  matrix  QTQ,  where  Q  is  the  data  matrix  formed  by  arranging 

N-l 

the  given  data  ( fm>n  in  columns  in  a  particular  fashion;  see  [1],  vol.  I, 
p.  2-2.  (An  equivalent  interpretation  is  that  QTQ  or  Q  are  approximated  by 
matrices  of  lower  rank,  i.e.,  singular  matrices.)  It  has  been  found  that 
the  weakest  eigenvector  of  Q'Q  is  less  dependent  on  the  absolute  noise  level 
and  can  furnish  more  useful  values  for  the  resonances  than  can  the 

inhomogeneous  solution.  Physically,  the  "best"  linear  prediction  of  a  noisy 
waveform  tends  to  zero,  whereas  an  eigenvector  can  maintain  all  its  compo¬ 
nents  nonzero,  regardless  of  the  absolute  noise  level.  At  present,  the 
weakest  eigenvector  solution  is  judged  to  be  the  best  of  all  iterative  and 
noniterative  methods  for  estimating  the  resonances  { )? ;  see  [1],  vol.  I, 

p.  2-28.  K  1 

When  the  number  of  resonances,  n,  in  (I)  is  unknown,  its  determination 

or  estimation  must  be  made  from  the  available  data  {f  if!"1.  If  k  is  the 

m  0 

true  (unknown)  number  of  resonances,  and  n  is  the  hypothesized  number, 
there  are  n-k  extraneous  resonance  estimates  produced.  A  maximum  likelihood 
procedure  developed  in  [1]  and  applied  to  the  z  smallest  eigenvalues  (for 
various  values  of  z)  has  been  found  to  give  reasonable  estimates  of  k.  An 
alternative  approach,  employing  time  reversal  of  the  data  sequence,  seems 
to  separate  extraneous  resonances,  but  more  study  is  suggested;  see  [1],  vol.  I, 
p.  3-26. 
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CONCLUSIONS 

The  usual  problems  associated  with  Prony's  method,  regarding  sensitivity 
to  noise,  have  been  attributed  to  dense  sampling  and  bias.  If  both  of  these 
problems  are  treated  properly  and  the  weakest  eigenvector  is  employed, 

Prony's  method  produces  excellent  estimates  of  the  resonances,  even  from  data 
with  high  noise  levels;  see  [1],  vol.  I,  p.  4-8. 

Studies  on  some  of  these  still -unanswered  questions  about  alternative 
procedures  for  order  selection  and  resonance  estimation  will  continue. 
Certainly,  further  improvements  in  the  procedures  and  performance  will  ensue. 
Applications  to  real  measured  data  have  yet  to  be  made,  however;  see  [1], 
vol.  I,  pp.  5-2  and  5-3. 
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Appendix  A 

A  MORE  GENERAL  MODEL 

Instead  of  (1)  of  the  main  text,  suppose  that  sequence  value 


n  p  _ 

gm  *  I  Ck  pj  +  I  0k  mpj  for  0  S  m  S  N-l, 
k=i  k-1 


where  p  can  be  larger  or  smaller  than  n.  Then  for  n+p  <  m  <  N-l, 
linear  prediction  error 


g„  -  la.  g  ,  -  Ip.  a  _  4 
ra  J  m  j  J  m  n  j 


j=l 


=  -  I  a  J  I  Ck  MJ"j  +  I  Ok(m-j)Mj"jl  I 

j=0  JLk=l  k=l  *  J 

-  I  pJ  I  C  pj"n'j  +  I  Ok(m-n-j)MJ'n‘j] 

j=l  JLk=l  *  k=l  J 

=  ‘  *  Ck  ^k  \  *  +  *  Pj  ^kn’jl 

k=l  K  KLj=0  J  K  j=l  J  K  J 

*  1  Dk  Mk  [  Z  “i(n'-j)u'kj  *  2  p .(m-n- j)pkn  Jl. 

k=l  K  k  Lj=0  J  k  j=l  J  J 


1) 


The  quantities  in  brackets  can  be  made  zero  for  n+p  <  m  <  N-l,  by 
both 

n  _  j  P  -  n-  j 

I  a.  p. J  +  I  Pi  H,,  J  =  0  for  1  <  k  i  n 


j=o  J  k  J=1  } 


and 


n  P 

I  a  .(m- j )mwj  +  2  p,(m-n 

j=0  J  J=i 


~j)Pkn  J  =  0  for  1  %  k  s  p. 


(A.  1) 

consider 


(A. 2) 

setting 

(A. 3) 

(A. 4) 
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This  combination  constitutes  n+p  linear  equations  in  the  n+p  unknowns  {a^ > ^ 
and  {6.-}?;  a  *  -1.  These  equations  can  be  put  in  the  form 

J  1  U 

«0  Mk+P  +  aiMk+P  1  +--*+  anMP  +  Pi^k"1  +---+  PD  =  0  for  1  *  k  *  n, 

(A. 5) 

<>ipJ+P  1  +• .  annMP  +  PiCn+DpjJ"1  +...+  Pp(n+p)  =  0  for  1  S  k  2  p. 

(A. 6) 

So  sequence  value  gm  can  be  determined  exactly  as  a  linear  combination  of 
its  previous  n+p  values,  for  n+p  <  m  <  N-l.  Notice  that  coefficients 

and  depend  on  {yk}°  (where  q  *  max(n,p)),  but  not  on  strengths  {C^}" 

or  {Dk)J.  See  also  [3],  pp.  174-175. 
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Appendix  B 

EIGENVECTOR  GENERALIZATION  OF  LINEAR  PREDICTION 


IDEAL  MODEL 

The  starting  point  Is  again  (1)  of  the  main  text.  We  now  generalize  (2) 
of  the  main  text  to  the  form 


n 

em  s  I  or.  gm  .  for  n  S  m  S  N-l,  (B.l) 

m  j_Q  J  nrj 


where  all  the  (a-jlg  are  arbitrary  for  the  moment.  It  follows,  from  substitu 
tlon  of  ( 1 )  of  the  main  text  in  (B-l),  that 


e  =  I 
m  j=0 


n  n  _  ,  n 
aj  Ck  Mk  =  1 


k=l 


P  m  " 

K  K  j=0 


'j  Mk 


n  ___  n  _  . 

=  2  C.  I  a  .  p"  J  for  n  S  m  2  N-l. 

k=l  K  *  j=0  J  * 


(B.2) 


Now  let  us  set 


n 


1 

j=0 


aQ  p£  +...+  an_^  Pk  +  =  0  for  1  £  k  £  n. 


(B.3) 


by  choice  of  {ai)g.  Since  there  are  only  n  equations  in  (B.3),  but  n+1 
unknowns,  we  will  not  get  a  unique  solution  for  the  (a.)g  unless  we  restrict 
them  somehow.  Also,  we  must  disallow  the  zero  solution. 

Observe  that  if  we  had  used  only  n  coefficients  {aj}g"*  in  (B.l),  we 
would  have  obtained,  instead  of  (B.3),  n  equations  in  n  unknowns.  However, 
the  only  solution  to  these  equations  is  the  zero  solution  a.  =  0  for  all  j, 
which  is  useless.  J 

Before  we  consider  the  restriction  on  {a.}«,  observe  that  substituting 
(B.3)  in  (B.2)  yields  J  u 
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n 

e  =  1  a.  g  .  =  0  for  n  2  m  2  N-l.  (B.4) 

m  j_0  J  3m-j 

That  Is,  we  can  find  an  Infinite  number  of  1  Inear  combinations  of  n+1  adja¬ 
cent  values  of  sequence  {9m>Q  generated  via  (1)  of  the  main  text, 

which  are  identically  zero  for  all  possible  locations  of  the  (n+l)-long 
average  within  the  record  of  length  N. 

Now  to  get  back  to  the  solution  of  (B.3)  for  the  coefficients  {“-In*  we 
observe  that  the  linear  predictive  approach  considered  in  (2)  et  seq.J 
of  the  main  text  amounts  to  choosing  oq  *  -1;  this  results  in  a  unique 
solution  for  the  n  linear  equations  (B.3)  in  the  remaining  n  unknowns  {am, 
and  is  called  forward  prediction  by  virtue  of  form  (5b)  of  the  main  text. 

An  obvious  alternative  would  be  to  select  aR  *  -1,  in  which  case  (B.3)  and 
(B.4)  would  yield  a  unique  solution  for  and 

Vr*oS.*-’VlWl  for  n  S  m  5  N-l.  (B.5) 


That  is,  we  are  doing  backward  linear  prediction  to  obtain  the  sequence 
values.  But  observe  that  both  of  these  cases  are  specializations  of  the 
linear  constraint 


on  the  coefficients 


where 


(B.6) 


(B.  7) 


are  column  matrices.  Constraint  (B.6)  prevents  the  zero  solution,  and  when 
combined  with  (B.3),  gives  a  unique  solution  for  A.  We  can  normalize  the 
matrix  of  constants,  C,  such  that 

CTC  3  1  (or  K  if  desired),  (B.8) 

without  loss  of  generality.  Forward  or  backward  prediction,  respectively,  cor¬ 
responds  to  choosing  all  the  { c . }q  equal  to  zero  except  for  edge  elements 
c0  or  cn,  respectively,  equal  toJ-l.  So,  generally,  we  can  realize  the  linear 
combi nati on. 
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I  a.  g  .  =  0  for  n  3  m  S  N-l,  (B.9) 

j_0  J  n»"J 

subject  to  {a Oq  satisfying  the  linear  constraint  (B.6),  which  guarantees  a 
nonzero  solution.  C  Is  any.  vector  satisfying  (B.8). 


ACTUAL  MEASURED  DATA 

N-l 

Now  consider  that  measured  data  (fm}Q  are  available.  Instead  of 
linear  prediction  (6)  of  the  main  text,  consider  the  more  general  linear 
combination  (as  In  (B.l)) 


n 

dm  s  I  a.  f  .  for  n  S  m  2  N-l,  (B.10) 

m  j=Q  J  ^  J 


where  set  (a.}g  Is  not  yet  specified.  Define  error  and  data  matrices 


d 

*  f  f  f 

n 

n  n-l  ‘  0 

dn+l 

,  F  = 

fn+l  •"  fl 

•  • 

•  • 

(N-n)x(n+l).  (B. 11) 

•  • 

_  Vl  fN-l-n_ 

Then  (B.10)  can  be  expressed  as 

D  =  FA  (B. 12) 

where  we  used  (B.7). 

Now  we  want  to  minimize  the  total  quadratic  error  of  (B.10),  namely, 


l  d£  =  DTD  =  ATFTFA  (B. 13) 

___  m 
m— n 


by  selection  of  A,  but  subject  to  linear  constraint  (B.6)  on  A,  which  guar¬ 
antees  a  nonzero  solution.  C  is  an  arbitrary,  yet-unspecif ied  matrix. 
Accordingly,  we  use  a  Lagrange  multiplier  2a  and  look  for  an  extremum  of 

ATS  A  -  2A  CTA,  (B. 14) 
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where  we  have  defined 


S  ~  F^F  (n+l)x(n+l)  matrix.  (B.15) 

S  Is  easily  seen  to  be  a  nonnegative  definite  matrix;  it  generally  has  full 
rank  when  N  >  2n.  Completing  the  square  in  (B . 14) ,  we  rewrite  it  as 

(A  -  XS_1C)T  S(A  -  XS_1C)  -  aVs“‘c.  (B.  16) 


The  extremum  is  then  obviously  realized  for  coefficient  matrix 


Aq  *  xS-1C. 


(B. 17) 


To  evaluate  x,  we  have  to  satisfy  the  linear  constraint  (B.6): 


T  -1 

AC'S  C  =  1, 


A  = 


_  1 


c's  C 


(B. 18) 


The  best  coefficient  set  is  then,  from  (B.17), 


C 

TT 


(B. 19) 


(Thus  the  best  coefficients  are  proportional  to  the  first  column  of  S”  for 
forward  linear  prediction,  or  to  the  last  column  for  backward  linear  prediction.) 
The  corresponding  minimum  value  of  the  total  quadratic  error.  (B.13),  is 


T  -i  -i 

c's  SS  C 
— T~n — r— 

(c's  C) 


_  1 
-  -T  il  • 

c's  C 


(B.20) 


(This  denominator  reduces  to  the  0,0  element  of  S"*  for  forward  linear 
prediction,  or  to  the  n,n  element  of  S“*  for  backward  linear  prediction.) 


But  this  result,  (B.20),  obviously  depends  on  the  particular  values 
assigned  to  the  constraint  vector  C  in  (B.6).  The  question  then  arises  as 
to  what  constraint  vector  would  yield  further  reduction  of  error  (B.20).  To 
determine  this,  let  matrix  S,  defined  in  (B.15),  have  eigenvalue  matrix 


A  = 


Ag  <  A ^ 


<. . .<  A 


(B.21) 
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and  modal  (eigenvector)  matrix 


(B.22) 


Then 

SE  -  EA  (B.23) 

or 

Se^  «  e^  for  0  <  k  <  n.  (B . 24 ) 

By  taking  the  Inverse  of  (B.23),  and  pre-  and  post-multiplying  by  E,  we 
obtain 


EA 


(B.25) 


or 


for  0  £  k  £  n  , 


(B.26) 


which  we  will  need  below.  The  inverse  matrix  has  the  same  eigenvectors  but 
the  inverse  eigenvalues  of  $. 

Now  any  n+1  column  matrix  can  be  expressed  in  terms  of  the  eigenvectors 
of  S.  In  particular,  suppose  we  let 


n 

C  =  Z  b.  e. 
k=0 


(B.27) 


Recalling  normalization  (B.8),  we  have  the  constraint  on  the  { }q : 

n  t  n  2 

Z  b.b0  e'e.  =  Z  b  =  1  ,  (B.28) 

k,2=0  *  *  *  *  k=0  K 

since  the  eigenvectors  are  orthonormal .  If  we  substitute  (B.27)  in 

(B.20),  the  denominator  is  given  by 

t  _  i  n  T  - 1  n  T  - 1 

C  S  C  =  Z  b  b  e'  S  e  =  Z  j.b  e  \  e 
k,£=0  k  2  k  2  k,£=0 


n 


k  j=0  "A  ».  =  k*0  VV 


(B.29) 
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where  we  employed  (B.26)  and  the  orthonormality  of  the  eigenvectors.  Now 
since  we  want  to  minimize  (B.20),  we  must  maximize  (B.29),  but  subject  to 
(B.28).  Obviously  the  best  choice  of  {bk>g  is  given  by 

bQ  =  ±  1,  bk  =  0  for  ISkSn,  (B  .30) 

where  xq  Is  the  smallest  eigenvalue  of  S;  see  (B.21).  Thus 

Minimum  total  quadratic  error  =  =  ^0  ’  ( B . 31 ) 

which  Is  the  smallest  eigenvalue  of  S  defined  in  (B.15). 

Now  we  can  employ  result  (B.30)  In  (B.27)  and  (B.19)  to  find  the  best 
coefficient  set  Ag.  We  have  C  *  +  eg,  and  (B.19)  becomes 


where  we  used  (B.26).  Thus  both  the  constraint  vector  and  the  best  linear 
weighting  of  the  data  in  (B.10)  are  equal  to  the  weakest  eigenvector  of  the 
matrix  S  =  FTF,  where  F  is  the  data  matrix  defined  in  (B.ll). 

We  can  now  return  to  (B.3)  to  solve  for  the  ( uk ^ 1 »  where  we  use  the 
components  of  the  weakest  eigenvector  of  S  for  the  that  is,  we  use 

e00 
e01 

e0n 


What  we  have  done  is  to  find  the  best  linear  constraint  such  that  the  total 
quadratic  error  (B.13)  is  minimized.  The  end  result  is  the  same  as  if  we 
had  minimized  (B.13)  directly,  subject  only  to  constraint 


A'A  =  I  a2.  =  1.  (B.34) 

j=0  J 

This  latter  interpretation  corresponds  to  the  best  A  vector  in  (n+l)-space, 
with  its  tip  on  the  unit  sphere,  that  minimizes  the  total  quadratic  error. 
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ON  GENERATION  OF  RANDOM  NUMBERS  WITH  SPECIFIED 
DISTRIBUTIONS  OR  DENSITIES 

INTRODUCTION 

The  generation  of  random  numbers  with  a  specified  probability  density 
function  or  a  specified  cumulative  distribution  function  is  a  frequent 
occurrence  in  the  simulation  of  signal  processing  techniques  that  are 
difficult  or  impossible  to  evaluate  analytically.  Accordingly,  it  is  of 
interest  to  be  able  to  generate  easily  and  efficiently  such  random  numbers. 
It  is  assumed  that  a  uniform  random  number  generator  is  already  available; 
that  is,  independent  random  variables  with  a  probability  density  function 
equal  to  unity  over  the  range  (0,1)  can  be  generated.  Validation  of  this 
assumption  for  a  particular  random  number  generator  must  be  confirmed  by 
numerical  tests,  several  of  which  are  tested  herein.  This  report  is  an 
extension  and  elaboration  of  [1]. 
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NONLINEAR  DISTORTION  OF  A  SINGLE  UNIFORM  RANDOM  VARIABLE 


Suppose  that  random  variable  x  is  uniformly  distributed  on  (0,1).  Let  the 
nonlinear  distortion  of  x  yield  the  random  variable  y  given  by 


where 


y 


(1) 


9i(  )  is  a  monotonically  increasing  function  of  its  argument, 

9d(  )  is  a  monotonically  decreasing  function  of  its  argument.  (2) 

This  restriction  of  the  nonlinear  distortion  to  be  monotonic  (either 
increasing  or  decreasing)  is  not  necessary;  it  is  done  here  only  for 
simplicity.  Then  since  random  variable  x  is  uniformly  distributed  on  (0,1), 
for  a  fixed  (nonrandom)  t  in  the  range  (0,1), 


g^x)  <  9i (t) 

t  =  Prob  |x  <  t^  « 

Prob  i 

or  , 

9d(x)  >  gd(t) 

|y  <  9i (t) 

\(9i(t)) 

=»  Prob  \  or  i 

1  =  < 

or 

[y  >9„(t)j 

}  -  Py(9d(t» 

where  we  used  (2),  (1),  and  denoted  the  cumulative  distribution  function  of 
random  variable  y  by  P  (  ).  we  rewrite  (3)  as 

t  =  Py ( 9-j  (t) ) 

or  (4) 

1  “  t  «  Py(gd(t))  , 

and  observe  that  cumulative  distribution  function  P  (  )  is  a  monotonically 
increasing  function  of  its  argument. 
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Now,  for  arbitrary  monotonic  function  h(  ) ,  we  denote  its  inverse  function 
by  h(  );  that  is,* 

h(h(u))  ■  u,  h(K(v))  ■  v  (5) 

Applying  monotonic  function  Py(  )  to  both  sides  of  (4),  and  using  (5),  there 
follows 


9-j (t)  -  ?y(t) 

or  (b) 

9dU)  “  ( 1  -  t)  . 

Employing  (6)  in  (1)  final ly  yields  the  random  variable 

fg,(x)  -  Py(*) 

y  .  g(x)  »  <  or  |  (/) 

[gd(x)  -  Py(l  -  x)  J 

as  the  desired  result  of  this  nonlinear  transformation  of  random  variaDle  x. 
If  cumulative  distribution  function  Py(  )  0f  random  variaDle  y  is  specified 
and  desired,  and  if  random  variable  x  is  uniformly  distributed  on  (0,1), 

(7)  tells  us  that  we  must  evaluate  the  inverse  of  the  given  cumulative 
distribution  function  and  use  it  as  the  nonlinear  transformation  of  either 
random  variable  x  or  random  variable  1-x,  depending  on  wnether  we  want  a 
monotonical  ly  increasing  or  monotonical  ly  decreasing  transformation, 
respectively.  The  key  element  to  this  approach  is  the  ease  with  wnich  the 
inverse  function  Py(  )  can  be  computed. 


*  Some  additional  relationships  among  functions  and  their  inverses  are 
presented  in  appendix  A. 
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EXAMPLES  OF  SINGLE-VARIATE  DISTORTION 

The  following  examples  have  been  adjusted  to  a  convenient  scale,  such  as 
zero  mean  or  unit  variance.  By  addition  of  a  constant  and/or  multiplication 
by  a  scale  factor,  alternative  desired  ranges  can  be  realized.  The  times  of 
execution  given  below  were  obtained  for  the  Hewlett-Packard  9845B  Desk 
Calculator  equipped  with  the  Fast  Processor  Upgrade  Kit;  they  include  the  time 
required  to  generate  the  uniform  random  variable  x.  Results  were  ootained  by 
averaging  over  1000  independent  trials.  Loop  counters  were  declared  INTEGER 
for  maximum  speed. 

EXPONENTIAL  DENSITY 

The  desired  probability  density  function  of  random  variable  y  is 
exponential*: 


Py(b)  =  exp(-u)  for  u  >  0 
The  corresponding  cumulative  distribution  function  is 


t 

P  (t)  =  f  du  p  { u )  =  1  -  exp(-t)  for  t  >  0 

J  J  J 


(8) 


(y) 


The  characteristic  function  of  random  variable  y  is  (for  future  reference) 


f,  (?) 


I 

•  «• 


du  exp(ifu)  p  (u)  =  (1  -  if) 


(10) 


In  order  to  find  the  inverse  function  to  (9),  we  let 


u  =  Py(t)  =  1  -  exp(-t)  .  (11) 

Solving  the  left  relation  for  t,  and  then  solving  the  outside  relation  for  t, 
we  get,  respectively, 


*  The  probability  density  functionsand  cumulative  distribution  functions  are 
zero  in  the  unspecified  regions;  for  example,  py(u)  =  0  for  u  <  0. 


4 


TR  6848 


t  -  Py(u),  t  -  -lr\(l  -  u)  . 

Combining  these  two,  there  follows  for  the  inverse  function 


(12) 


fSy(u)  ■  -Jtn(l-u)  for  0  <  u  <  1  .  (13) 

The  nonlinear  transformation,  by  reference  to  (7),  is  then 


y 


f -Jtn(l-x) 

1  °r 
L-Jtn  * 


(14) 


The  time  of  execution  for  generation  of  a  random  variable  y  via  (14)  is 
1.9  msec. 


RAYLEIGH  DENSITY 

? 

py(u)  «  u  exp(-u  / 2)  for  u  >  0  , 

2 

Py(u)  »  1  -  exp(-u  12)  for  u  >  0 
Via  a  procedure  similar  to  (11)  and  (12),  there  follows 

Py(u)  =  ^ j -2  in(l-u)'  for  0  <  u  <  1 
The  nonlinear  transformation  that  yields  Rayleigh  random  variables  is 


V-2  Jtnx  ‘ 


The  time  of  execution  of  (17)  is  2.5  msec. 


(lb) 


(16) 


(17) 
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GAUSSIAN  OENSITY 

Py(u)  -  (2w)_1/2  exp(-u2/2)  for  all  u, 

Py(u)  -  jf  dt  (2w)'1/2  exp(-t2/ 2)  z  f(u)  ,  (18) 

-tO 

fy(f)  -  exp  (-f2/2)  . 

The  inverse  function  is  [2;  26.2,  26.2.22,  26.2.23] 

Py(u)  ■  |(u)  ■  -|(l-u)  for  0  <  u  <  1  .  ( 1 9 ) 

The  nonlinear  transformation  that  yields  Gaussian  random  variables  from 
uniformly  distributed  ones  is  then,  from  (7)  and  (19), 


The  time  of  execution  of  (20)  is  13.2  msec. 

A  much  better  approach,  for  this  particular  case  of  generation  of  Gaussian 
random  variables,  is  as  follows:  let  x^  and  X£  be  two  independent  random 
variables,  uniformly  distributed  on  (0,1).  Then  according  to  the  previous 
example,  the  two  independent  random  variables 

r  «Y“2  in  x^  ,  e  =  2w  x2  ,  (21) 

have,  respectively,  the  probability  density  functions 

2 

Pr(u)  =  u  exp(-u  12)  for  u  >  0  , 

P_(u)  =  (2tt)_1  for  0  <  u  <  2n  .  (22) 
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Now  define  two  new  random  variables  by  the  nonlinear  transformations 


«  r  cos  o,  ^2  *  r  s’n  ® 


(23) 


The  joint  characteristic  function  of  yj  and  y2  is  [2;  y .1 .21  and  11.4.29] 
E{exp(ify1  ♦  ipy2)}  -  E[exp(ifr  cos  e  +  ipr  sin  ejj 

■  JJ*du  dv  pr(u)  Pe(v)  exp  (ifu  cos  v  ipu  sin  v) 

-o© 

+•  2 

»  j^du  u  exp(-u2/2)  ^  dv  ^  exp(iu  (J  cos  v  +  p  sin  y)) 

■  J  du  u  exp(-u2/2)  JQ(u^  +  p2^ 


-  exp(-  \f  -  \  u2) 


(24) 


Thus  yj  and  y2  are  independent  Gaussian  random  variables,  each  with 
zero-mean  and  unit  variance.  The  time  of  execution  of  (21)  and  (23)  is 
5.4  msec  per  random  variable  (actually  10.7  msec  for  a  pair  of  independent 
Gaussian  random  variables.)  This  5.4  msec  is  considerably  less  than  the 
13.2  msec  required  of  (20),  which  also  requires  a  special  function 
definition.  A  more  general  distortion  than  (21 )— (2 3 )  is  considered  in 
appendix  B. 


Another  alternative  for  the  generation  of  approximately  Gaussian  random 
variables  is  to  sum  N  independent  random  variables,  uniformly  distributed  over 
(0,1).  By  subtraction  of  the  constant  N/2  and  scaling  by  Y12/N',  a  zero-mean 
unit-variance  random  variable  can  be  generated  which,  however,  is  limited  to 
the  finite  range  (-V3fT,  )•  A  table  of  execution  times  and  range  values  is 

given  below. 


7 


TR  6843 


Table  1.  Execution  Times  for  Sum  of  N  Independent  Random  Variables 


N 

Execution  Time  (msec) 

Range  of  Values 

4 

2.8 

-3.46,  3.46 

5 

3.3 

-3.87,  3.87 

6 

3.9 

-4.24,  4.24 

7 

4.4 

-4.58,  4.58 

8 

4.9 

-4.90,  4.90 

9 

5.4 

-5.20,  5.20 

10 

5.9 

-5.48,  5.48 

20 

11.0 

-7.75,  7.75 

30 

16.2 

-9.4y,  y.4y 

The  characteristic  function  of  this  random  variable  is  [sin(Rf)/(Rf)]N  where 
R  «  yj/f? ,  and  the  normalized  fourth-cumulant  of  the  random  variable  generated 
by  this  summation  procedure  is  -1.2/N.  If  the  non-Gaussianness  can  be 
tolerated,  this  summation  procedure  is  then  a  viable  alternative  to  (21)  and 
(23)  in  terms  of  execution  time,  for  N  <  9. 

CAUCHY  DENSITY 

Pv(u)  *  7 - for  all  u, 

y  1  ♦  \ic 

Py(u)  =  \  +  7  arc  tan(u), 

fy(?)  -  exp  (- |yj  >  .  (25) 

This  is  the  probability  density  function  of  the  ratio  of  two  zero-mean 
independent  Gaussian  random  variables.  The  inverse  function  to  the  cumulative 
distribution  function  is 


Py(u)  =  tan  ^u  -  yjj  for  0  <  u  <  1  ,  (2b) 

and  the  nonlinear  transformation  is 
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[  £  •  ?)] 
[\  -  fil 


The  execution  time  of  (27)  is  2.8  msec.  Equivalently,  the  transformation 
tan(*x)  would  realize  the  desired  probability  density  function,  although  it  is 
not  monotonic  in  x  over  (0,1). 


RECTIFIED  CAUCHY  DENSITY 


Pv(u)  -  7  --1  -  y  for  u  >  0  , 

y  *  i  +  u‘ 

2 

P  (u)  «  -  arc  tan(u)  for  u  >  0  , 

J  ^ 

Py(u)  -  tan  uj  for  0  <  u  <  1  , 

fta"  (f  x)') 

y  -<  or  ►  .  (28) 

cot  xj 

The  execution  time  of  (28)  is  2.7  msec. 
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REALIZATION  OF  DISTRIBUTION  VIA  COMBINATION  OF  SEVERAL  RANDOM  VARIABLES 

If  several  independent  random  variables  are  added,  their  characteristic 
functions  multiply.  So  if  a  specified  probability  density  function  or 
cumulative  distribution  function  has  a  characteristic  function  which  can  be 
broken  down  into  easily  realizable  terms,  this  observation  can  be  utilized  to 
generate  complicated  distributions  with  relative  ease.  For  example,  consider 
the  following. 

CHI-SQUARED  DENSITY  WITH  2N  DEGREES  OF  FREEDOM 

This  variate  is  normally  thought  of  as  being  generated  by  summing  the 
squares  of  2N  zero-mean  unit-variance  independent  Gaussian  random  variaDles. 
One  half  of  this  sum  has  the  probability  density  function 


Py(u) 


uN_1  exp(-u) 

"TfTTTl - 


for  u  >  0 


(2y) 


The  cumulative  distribution  function  is 

N-l 

Py(u)  *  1  -  exp(-u)  u  / n!  for  u  >  0  .  (30) 

n*0 

The  inverse  function,  Py(  )t  to  (30)  is  not  available  in  closed  form  for 
N  >  2,  so  that  recourse  to  (7)  is  not  reasonable. 


However,  the  characteristic  function  corresponding  to  (29)  is 


V?)  «  (1  -  i?)"N  •  (3D 

But  this  is  (10)  multiplied  by  itself  N  times.  Then  (14)  reveals  that  we  can 
generate  y  according  to  a  sum  of  N  independent  variates: 


(32) 


The  latter  form  in  (32)  is  preferable  computationally;  it  involves  N-l 
multiplies  and  only  one  logarithm,  and  is  obviously  much  quicker  than 
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generating  2N  Gaussian  random  variables'and  summing  their  squares.  The  time 
of  execution  of  (32)  is  given  below. 

Table  2.  Execution  Times  for  Chi-Squared  Variate  of  2N  Degrees  of  Freedom 


N 

Execution  Time  (msec) 

10 

8.0 

20 

14.1 

30 

20.2 

If  we  were  to  use  the  nonlinear  distortion  procedure  in  (20)  for  generating 
Gaussian  random  variables,  and  square  and  add  2N  such  numbers  to  generate  a 
Chi-squared  variate,  the  time  of  execution  would  be  2N  (13.2)  msec.  For  the 
examples  in  table  2,  these  times  are  264,  528,  and  792  msec  respectively, 
which  are  far  greater  than  the  execution  times  for  (32). 


A  closely  related  random  variable  to  (32)  is 


r 


*g(y) 


The  probability  density  function  of  random  variable  r  is 


(33) 


Pr (u)  =  py(1(u))  $(u)  =  py(u2/ 2)  u 

•  - - exp^~U  /2^  for  u  >  0  .  (34) 

2N_1  (N-l)  .* 

The  cumulative  distribution  function  of  r  is 

Pf(u)  =  1  -  exp(-u2/2)  ^  for  u  >  0  .  (3b) 

n=0 
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The  time  of  execution  of  (33)  is  given  in  table  3. 

Table  3.  Execution  Times  for  Square  Root  of  Chi-Squared  Variate 


N 

Execution  Time  (msec) 

10 

8.6 

20 

14.7 

30 

20.8 

Another  example  of  this  combination  of  variates  is  furnished  by  the 
following. 

Q  DISTRIBUTION  AND  RICE  DENSITY 


Suppose  that  y^  and  yg  are  two  independent  zero-mean  unit-variance 
Gaussian  random  variables,  as  generated  via  (21)  and  (23).  Then  for  a 
constant  d,  the  random  variable 


has  characteristic  function  [2;  9.1.21  and  11.4.29] 

fz(F)  »  E{exp(ijzj}  =  E jexp 

+at  2  7t 


(3o) 


J  du  u  exp(-u^/2)  exP^  (d^  +  u^)j  ^  dv 
exp( if d2/ 2)  J  du  u  exp£  (1-i? JQ(f  d 


(1-if)"1  expfe- 


_!l_\ 

i-i?; 


^  exp(i  Jd  u  cos  v) 
u) 


(3/) 
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The  corresponding  probability  density  function  is 


pz(u)  -  exp^  j-  -  u)  I0(dV2l?)  for  u 


>  0  , 


as  may  be  confirmed  by  direct  Fourier  transformation  (according  to  (10))  of 
(38);  see  [2;  11.4.29].  The  cumulative  distribution  function  of  z  is 


P2(u)  -  1  -  Q(d,V2l?  )  for  u  >  0  , 


where  Marcum's  Q  function  is  defined  as 


Q(a,b) 


dx  x  exp 


(-*H> 


Vax> 


The  time  of  execution  of  (36)  and  (21)  is  7.2  msec;  there  is  no  need  to  employ 
(23). 

A  closely  related  random  variable  to  (36)  is 


¥?r-  (c 


d  +  2dr  cos  e 


*  r*f  . 


The  probability  density  function  of  y  is  the  Rice  density 


Py(u)  *  P2(u  /2)  u 


u  exp^-  - — IQ(du)  for  u  >  0  , 


and  the  cumulative  distribution  function  of  y  is 


P  (u)  =  1  -  Q(d,u)  for  u  >  0 


The  time  of  execution  of  (41)  and  (21)  is  7.5  msec;  there  is  no  need  to  employ 
(23). 
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These  two  examples,  (36)  and  (41),  would  again  be  analytically  very 
difficult  to  realize  by  use  of  (7),  because  the  inverse  functions  to 
cumulative  distribution  functions  (39)  and  (43)  are  not  available  in  closed 
form.  Furthermore,  different  values  of  d  are  easily  accommodated  in  (36)  and 
(41),  whereas  the  inverses  to  (39)  and  (43)  would  necessarily  involve  d  as  a 
parameter. 

A  more  general  situation  is  encountered  for  the  following. 

Qm  DISTRIBUTION 

This  example  is  a  combination  of  the  last  two.  Let,  as  in  (36)  and  (21), 
v  *  7  i>i  +  d^2  +  y2]  *  +  2dr  cos  e  +  r2]  ,  (44) 


where 


r  =y~2  in  x^  ,  e  .  2»  x2 
And  as  in  (32),  let 


(45) 


w  =  -in 


(46) 


f 

All  the  random  variables,  |xmJ  ,  are  independent  and  uniformly  distributed 
on  (0,1).  Then  let  random  variable  z  be  the  sum  of  the  above  two: 


1  ?  of  M'*'! 

z  =  v+w  =  £[d  +  2dr  cos  e  ♦  r  ]  -  in<  “FT  xm 

[m=3 

The  characteristic  function  of  random  variable  z  follows  upon  use  of 
(37)  and  (31): 


(47) 
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fz(?)  -  exp(jr  •  (48) 

The  corresponding  probability  density  function  and  cumulative  distriDution 
function  for  z  are 


where  the  QM  function  is  defined  by  [3] 

QM(a,b)  -  jfdx(l)  x  exp ^  ^I^Ux)  .  (50) 

The  execution  time  of  (45)  and  (47)  is  given  in  table  4  for  several  values  of 
M.  The  use  of  (7)  would  have  required  the  inverse  of  (49),  a  rather 
formidable  analytical  task  involving  the  two  parameters  d  and  M. 


Table  4.  Execution  Times  for  QM  variate 


M 

Execution  Time  (msec) 

10 

14.6 

20 

20.6 

30 

26.7 

A  closely  related  random  variable  to  (47)  is 


y  =y2F  = 


?  ? 

'ri+1 

i- 

d  ♦  2dr  cos  e  +  r -  2  Jn  • 

1 U 


(51) 
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There  follows 


M“1  /  2  2\ 

Py(u)  -  exp^-  d  l  —J  I^du)  for  u  >  0  , 

P  (u)  »  1  -  QM(d,u)  for  u  >  0  .  (52) 


The  execution  times  for  (45)  and  (51)  were  0.4  msec  larger  than  those  given  in 
table  4. 
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PRODUCTS  OF  RANDOM  VARIABLES 


We  can  now  take  products  of  some  of  the  random  variables  above  and 
generate  additional  cases  of  complicated  probability  density  functions.  For 
example,  if  we  take  the  product  of  two  random  variables  as  given  by  (32), 


Z  “  yly2 


(53) 


where  |x^j  and  jx^j  are  uniformly  distributed,  the  probability  density 
function  of  the  product  random  variable  is,  from  (29)  and  [4;  3.471  9J, 


Pz(u)  - 


I 


dt 

itr 


py,(t)  pJt) 


¥  N1-N2-l  N2-l 

)  dt  pi,-!)!  ( n2-i ) ;  exp 

0 


(-*-  r) 


VN2 


-1 


2  u 


mp-T]'!  TnpTTT  for 


u  >  0 


(54) 


Here  Kv(  )  iS  a  modified  Bessel  function  of  the  second  kind  and  order  v 

[2;  section  9.6];  it  is  even  in  v  [2;  9.6.6].  Execution  times  for  (53)  can  be 

determined  from  (32)  and  table  2. 


The  random  variable  w  *  2z*^,  wnere  z  is  given  by  (53),  has  the 
probability  density  function 


PW(U) 


N, +N?-1 

2(u/2)  k  [u) 

(Nri):  (n2-d:  KNrN  (u) 


for  u  >  0 


(5b) 


The  product  of  two  independent  zero-mean  Gaussian  random  variables,  as 
given  by  (21)  and  (23),  is  given  more  simply  by 
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2  12 

z  -  y^  -  r  sin  9  cos  e  ■  j  r  sin  (2o)  *  -Xn(x^)  s i n( 4ttX2 )  •  (5b) 

The  probability  density  function  of  z  can  be  determined  exactly  as  in  (54), 
with  the  result 


pz(u)  *  w  Ko(lu|  )  foral,u  •  (57) 

The  4*  sweep  of  the  sin  argument  in  (56)  is  unnecessary;  the  following  will 
accomplish  the  same  probability  density  function: 

z  -JMxj)  cos(wx2)  ,  (58) 

where  Xj  and  ^  are  uniform  over  (0,1).  The  time  of  execution  of  (58)  is 
6.0  msec. 
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GENERATION  FROM  A  NON  UNIFORMLY  DISTRIBUTED  RANDOM  VARIABLE 

Suppose  that  It  is  possible  to  generate  a  random  variable  x  with 
cumulative  distribution  function  Px(  )t  and  that  the  cumulative  distribution 
function  of 


y  -  9(x)  (59) 

is  desired  to  be  P  (  ).  Then  for  g  ■  g^,  a  monotonical ly  increasing 
function,  the  cumulative  distribution  function  of  x  is 

Px(u)  -  Prob  {x  <  u}  -  Prob  f gi (x)  <  g i ( u )} 

-  Prob  fy  <  gi(u)J  «  Py(gi(u) )  .  (bO) 

Applying  inverse  function  Py  to  both  sides  of  (60),  the  desired  nonlinearity 
is 

gi(u)  -  Py(Px(u))  .  (6i) 

Equation  (7)  is  obviously  the  special  case  of  this  when  x  is  uniformly 
distributed.  Alternative  expressions  to  (61)  are  available  by  use  of  some  of 
the  relations  in  appendix  A. 

If  we  want  to  use  distortion  g  =  gd>  a  monotonical ly  decreasing 
function,  to  generate  y,  we  have 

Px(u)  =  Prob  [x  <  u}  =  Prob  fgd(x)  >  gd(u)J 

=  Prob  [y  >  gd(u)}  =  1  -  Py(gd(u))  .  (62) 

Inverting  this  relation, 

9d(u)  *  Py(l  -  px(uj)  •  (03) 
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This  also  reduces  to  (7)  for  a  uniformly  distributed  random  variable  x. 
In  either  case,  (61)  or  (63),  the  inverse  function  of  the  desired  cumulative 
distribution  function,  P  must  be  realized.  Then  a  cascade  operation  as 
dictated  by  (61)  or  (63)  must  be  employed.  The  desired  random  variable  is 
given  by 


I 
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•  TESTS  OF  UNIFORM  RANDOM  NUMBER  GENERATOR 

A  critical  component  of  the  procedures  described  above  is  the  generator  of 
random  variables,  {xn},  with  a  f irst-order  probability  density  function  that 
is  flat  over  (0,1), and  with  statistically  independent  samples.  Here  we  will 
investigate  several  tests  of  the  random  number  generator,  RND,  of  the 
Hewlett-Packard  9845  Desk  Calculator.  Inherent  in  this  investigation  is  the 
need  to  state  the  confidence  associated  with  a  particular  measurement  or 
estimate;  for  example,  see  [5]. 

CORRELATION  TEST 

The  sample  correlation  of  a  set  of  N  measurements  fyn3^  is  defined 
here  as 


R 


k 


N-k  ynyn-k 


for  0  <  k  <  N 


(65) 


We  presume  that  the  {yn3  are  all  independent  with  a  flat  probability  density 
function 


Pv(“)  -  -7=  for  |u|  <VT  .  (6b) 

y  2V? 

These  random  variables  can  be  obtained  from  uniform  {xn}  according  to  the 
linear  transformation 

-  ?)  ;  (67) 

they  have  the  convenient  normalized  values 


E(yn)  =  0  ,  E(y^)  =  1  .  (68) 

More  generally,  from  (66), 
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Under  these  assumptions,  it  is  readily  shown,  by  use  of  (69),  that  Rk  is 
unbiased,  that  is 


and  that  the  standard  deviation  of  Rk  is 


a(Rk) 


(1.25N)"1/2  for 
(N-k)‘1/2  for 


k  «  0  / 

1  <  k  <  N  ( 


(71) 


Thus  we  expect  Rk  to  lie  within  2  or  3  standard  deviations,  c(Rk),  of 
E(Rk)  most  of  the  time;  that  is,  the  normalized  random  variable 


Rk  -  E(Rk) 
rk  ■  — 


(72) 


should  be  between  ±  2  most  of  the  time,  with  rare  excursions  to  +  3,  if  tne 
K)  are  truly  independent  [5].  In  table  5,  the  results  of  sample  runs  for 
N.100,  1000,  and  10000  are  listed.  They  furnish  no  reason  for  rejecting  the 

hypothesis  that  [xnj  are  independent.  Runs  for  other  sets  of  random  numDers 
yielded  results  very  similar  to  table  5.  An  alternative  test  on  the  whiteness 

°f  (xnl  is  9iven  appendix  C. 

Table  5.  Sample  Correlation  Results  for  (72) 

Delay  k  rk  for  N  =  100  r^  for  N  *  1000  rk  for  N  =  10000 


0 

1.64 

0.78 

0.65 

1 

-1.04 

-0.91 

0.25 

2 

-0.26 

-0.68 

-0.02 

3 

-0.10 

1.39 

-0.1.5 

4 

1.44 

0.31 

0.27 

5 

-2.31 

-0.30 

0.47 

6 

-0.49 

-1.20 

-0.20 

7 

0.83 

0.81 

-1.40 

8 

0.17 

0.61 

-1.47 

9 

-0.61 

-0.77 

-0.91 

10 

-1.43 

-0.45 

0.45 
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MOMENT  TEST 


The  Sample  moment  of  order  k  for  a  set  of  N  measurements  {yn]  is 


1  4  k 
“k  ■  n  ^1  y° 


The  mean  and  variance  of  uk  are 

e(f*k)  -  e(y„k).  »"•(»„) -b  ^(ynk)  • 

and  can  be  obtained  from  (66)  and  (69).  The  random  variable 


(73) 


(74) 


-  E(wk) 

"V  ■  JET:  Dev. (wk)  (75) 

should  therefore  lie  mostly  in  the  range  +  2.  A  sample  result  is  given  in 
table  6,  which  again  confirms  the  flat  probability  density  function  hypothesis 
in  (66).  (Although  these  particular  runs  all  resulted  in  positive  numbers, 
other  sample  runs  resulted  in  a  preponderance  of  negative  values  for  mk.) 

Table  6.  Sample  Moment  Results  for  (75) 


Moment  k 

m.  for  N  «  100 
k 

m^  for  N  *  1000 

mk  for  N  x  10000 

1 

1.07 

1.37 

0.52 

2 

1.64 

0.78 

0.65 

3 

1.08 

1.09 

0.42 

4 

1.49 

0.67 

0.89 

5 

0.95 

1.11 

0.30 

6 

1.21 

0.51 

0.94 
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FIRST -ORDER  DISTRIBUTION  TEST 


Let  Ik  be  an  interval  of  the  line  segment  (0,1).  Then  let  cn  be  a 
counting  variable  given  by 


c 


n 


1  if  x„  t  I 
<  n 

0  otherwise 


for  1  <  n  <  N 


Then  an  estimate  of  the  probability  Pk,  that  xn  e  Ik,  is  given  by  the 
quantity 


(76) 


This  random  variable  has  mean  and  variance 

E{V  *  Pk’  Var(V  “  Pk(1"Pk)/N 

Thus  the  random  variable 


(77) 


(78) 


^k  ~  ^k 

V  .  - - - =77 r  (79) 

k  [pka-pk)/f* 

should  lie  most  often  in  the  region  +  2  if  xp  truly  does  have  probability 

Pk  of  lying  in  Ik.  In  table  7  is  displayed  tne  results  of  sample  runs  for 
the  case  where  interval  (0,1)  is  broken  into  10  equal  parts;  that  is, 

*k  *  (^W  ’  for  1  i  k  <  10  •  (a°) 

Once  again,  there  is  no  reason  to  reject  the  hypothesis  that  the  random  number 
generator  is  uniformly  distributed  over  (0,1). 
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Table  7.  Sample  Probability  Results  for  (79) 


Interval  k 

for  N  -  1000 

< 

-h 

O 

“I 

H 

i—4 

O 

o 

o 

o 

vk  for  N  *  100U00 

1 

-0.74 

0.67 

0.50 

2 

1.16 

-0.57 

0.b7 

3 

-1.26 

-1.13 

-1.02 

4 

-1.37 

0.03 

-0.96 

5 

-0.32 

0.90 

-1.26 

6 

-0.32 

-0.73 

-0.64 

7 

1.48 

-0.27 

0.53 

8 

-0.53 

-0.13 

-0.32 

9 

0.32 

-0.07 

1.87 

10 

1.58 

1.30 

0.64 
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SUMMARY 

Several  methods  of  generating  random  variables  with  specified  cumulative 
distribution  functions  have  been  presented  and  evaluated  in  terms  of  their 
time  of  execution  and  efficiency  of  generation.  They  include  nonlinear 
distortion  of  a  single  (uniformly  distributed)  random  variable  or  through 
combinations  of  simply  generated  random  variables.  The  former  approach 
requires  the  ability  to  realize  the  inverse  function  to  the  desired  cumulative 
distribution  function,  whereas  the  latter  approach  is  very  fruitful  if  the 
characteristic  function  corresponding  to  the  specified  cumulative  distriDution 
function  can  be  broken  down  into  a  product  of  simpler  components.  Which 
approach  to  adopt  depends  on  the  particular  example  and  the  exact  way  that  any 
parameters  enter  into  the  cumulative  distribution  function.  Of  course,  tne 
inverse  to  a  cumulative  distribution  function  can  always  be  evaluated 
numerically  and  used  as  a  table  look-up,  in  order  to  generate  transformed 
random  variables;  however,  this  numerical  procedure  would  have  to  be  repeated 
if  any  parameter  of  the  cumulative  distribution  function  were  desired  changed. 

The  ability  to  generate  uniformly  distributed  random  variables  is  a  key 
component  of  this  procedure.  Several  statistical  tests  on  the  Hewlett-Packard 
9845  random  number  generator  have  confirmed  it  to  have  a  flat  probability 
density  function  and  independent  samples. 

These  techniques  are  useful  for  generating  sets  of  random  variables  of 
size  N  with  a  specified  cumulative  distribution  function,  and  then  plotting 
the  sample  cumulative  distribution  functiorvas  described  in  [6],  in  order  to 
ascertain  the  amount  of  fluctuation  which  is  typical  for  that  set  size  N  and 
in  different  regions  of  probability.  Then  by  comparing  the  amount  of 
fluctuation  of  a  measured  data  set  (of  unknown  statistics)  with  typical  sample 
cumulative  distribution  functions  of  the  same  size  and  with  a  known  specified 
cumulative  distribution  function,  decisions  on  acceptance  or  rejection  of  a 
hypothesized  cumulative  distribution  function  can  be  made  witn  confidence. 

See  [6,  figures  6  and  7]  for  illustrations  of  this  procedure. 
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APPENOIX  A.  SOME  INVERSE  FUNCTION  RELATIONS 
In  this  appendix,  x  and  y  are  real  (non  random)  variables.  Let  g(x)  be  a 


monotonic  function  for  x  in  a  given  range,  and  let 

y  -  g(x)  .  ( A— i ) 

The  inverse  relation  to  (A— 1 )  is,  for  y  in  the  appropriate  range, 

x  -  g(y)  .  (A-2) 

Now  suppose  that  we  cascade  nonlinearities:  let 

z  «  h(y)  .  h(g(x) ) s  f(x)  ,  (A-3) 

to  yield  overall  nonlinearity  f.  Then  the  outer  equality  in  (A-3)  yields 

x  -  f(z)  ,  ( A— 4 ) 

whereas  the  first  and  third  terms  in  (A-3)  yield 

h(z)=g(x),  and  g(fi(z))«x  .  (A-5) 


Combining  (A— 4 )  and  (A-5),  there  follows  for  the  inverse  function  of  the 
cascade,  (A-3), 

f(z)-g(h(z))  ,  (A-b) 

in  terms  of  the  inverses  of  the  individual  transformations. 

If  we  combine  (A-l)  and  (A-2),  we  get 

y  =  g(§(y))  .  (A-7) 

Taking  a  derivative  with  respect  to  y,  we  find 
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l  -  g'(g(y) )  g'(y)  ; 

•  (A'6) 

That  is,  the  derivative  of  the  inverse,  g,  can  be  found  in  terms  of  the 
Inverse  function  and  the  derivative  of  the  original  function  g. 

Suppose  that  y  is  given  in  terms  of  x  via  transformation 

y-g(?W)  ,  (A-9) 

but  the  inverse  function  f  is  impossible  or  very  difficult  to  obtain  from 
given  function  f.  A  simple  way  of  evaluating  y  vs  x,  then,  is  parametrically 
by  letting 


t  -  f(x),  to  get  x  -  f(t),  y  -  g(t) 


( A— 10 ) 


Now,  as  t  is  varied,  f  and  g  can  be  evaluated  to  determine  y  vs  x.  This 
transformation  also  allows  evaluation  of  an  integral  involving  an  inverse 
function: 


b  ib 

Jdxg(?(x)).  \  dt  f'(t)  g(t)  ,  (A-li) 


where 


ta  =  f(a),  tb  =  7(b)  .  (A-li?) 

Another  use  of  inverse  functions  in  integral  evaluation  is  afforded  by  the 
example 


I 


b 

J  dx  g( x)  w( x ) 
a 


( A— 1  J) 
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where  function  w(x)  need  not  be  monotonic  or  possess  an  inverse.  g(x)  is 


assumed  monotonic  in  (a,b).  Letting 

y  -  g(x),  x  -  g(y),  dx  -  dy  g'(y)  ,  (A-l4) 

there  follows,  for  the  integral  in  terms  of  inverse  function  §, 

9(b) 

i  *  ^  dy  y  ?'(y)  w(3(y) )  .  ( a-15) 

g(a) 

Integrating  by  parts,  using  identification 

u  -  y  w(g(y)),  v  «  g(y)  ,  (A-16) 

we  get  the  alternative  form 

g(b) 

I  -  b  g(b)  w(b)  -  a  g(a)  w(a)  -  J  dy  g(y)  ^  {y  w(g(y))}  .  (A-I7) 

g(a) 

The  special  case  of  w(x)  »  1  in  ( A— 1 3 )  and  (A-17),  namely 
d  g(b) 

j  dx  g ( x )  .  b  g(b)  -  a  g( a)  -  j"  dy  g(y)  ,  (A-18) 

a  g(a) 


has  the  geometrical  interpretation  in  figure  A-l.  Equation  (A-18)  is  the 
statement  that  A^  +  A2  ♦  A3  *  total  area  b  g(b). 

As  an  example  of  the  application  of  (A-l 3 )  and  (A-15),  consider 

g(x)  =  arc  Sin(x),  g(y)  =  sin  y  .  (A-ly) 

There  follows 

b  arc  sin(b) 

^  dx  arc  sin(x)  w(x)  =  ^  dy  y  cos (y)  w(sin  y) .  (A-20) 

a  arc  s  in(a) 
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So  if  w  is  a  polynomial,  the  integral  on  the  right  side  of  (A-20)  can  be 
evaluated  in  closed  form. 
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APPENDIX  B.  A  MORE  GENERAL  DISTORTION 

In  (21),  a  random  variable  r  with  a  Rayleigh  probability  density  function 
and  a  random  variable  e  with  a  uniform  probability  density  function  were 
generated.  These  were  then  used  in  nonlinear  transformation  (23)  to  generate 
two  independent  Gaussian  random  variables  y^  and  y2.  Here  we  will 
generalize  the  probability  density  function  pr  of  r  in  (22),  and  allow  pr 
to  be  arbitrary,  e  is  still  uniform  over  2*. 

The  two  new  random  variables  generated  are  again  as  in  (23): 


yi  -  r  cos  e,  y2  -  r  sin  e  (B-l) 

Because  of  the  uniform  distribution  of  ©  over  2w,  the  joint  probability 
density  function  of  y^  and  y2  is  of  the  symmetric  form 


P(yj.  y2)  -  for  all  yv  y2 

To  determine  h,  observe  that,  for  t  >  0, 

P  *  Prob  {‘fi  *  »r<  *)  s  Ifdyl  dy2  p(yl’  y2} 


(B-2) 


fj*  dyj  dy2  h  ^y^  "♦  y2'^  =  j  dp  p  |  d^  h(p)  -  2w  ^dp  p  n(p)  ,  ( B-3 ) 

C*  0  -IT  ^ 


where  Ct  iS  a  circle  of  radius  t  centered  at  the  origin.  But  also,  from 
(B-l), 


P  =  Prob  [r  <  t]  =  jdu  p  (u) 

1  o 

Equatinq  (B-3)  and  (B-4)  and  taking  a  derivative  with  respect  to  t,  there 
f  o  1  lows 


(B-4) 


pr(t) 

h(t)  =  ^ —  for  t  >  0 


(B-5) 
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Reference  to  (B-2)  then  yields  for  the  joint  probability  density  function  of 

*1.  *2. 


p(yj»  y2) 


2’Vyl  *  *2 


for  all  ylt  y2 


( B— 6 ) 


yi  and  y2  are  statistically  dependent  in  general. 

EXAMPLE  1. 

Our  first  case  is  the  probability  density  function  in  (34),  for  random 
variable  r  as  generated  by  (33): 


p  (u)  _  u2N  1  exp(-u2/2)  fQr  u  >  0  # 

r  2N-i(N-l).* 


(B-7) 


Substitution  in  (B-6)  yields  the  joint  prooability  density  function 


2  2 

p(yr  y2)  *  |2ir(  N-l)  |J -1  ^  *2 


lN-1 


2  .  2 
yl  +  *2 


2 — j  exp  i - 2 — /  for  a11  y\>  y2  •  (B~8) 


The  special  case  of  N*1  reduces  to  the  pair  of  Gaussian  ranaom  variables 
already  considered  in  (2 1 )— (24) .  For  N=2,  (B-7)  and  (B-8)  yield 


Pr(u)  =  \  u3  exp(-u2/2) 


2  2 
yl  +  H 


2  2 ' 

*1  +y2 


p(yi»  ^  -  — ^exp  [ — 7 


(B-9) 


EXAMPLE  2. 

Here  random  variable  r  is  generated  according  to  (32),  and  nas  the 
probability  density  function  given  by  (29): 


N-l 

/  \  u  exp(-u)  ,  n 

pr(u) - (H-l)j  for  u  >  0 


(B-10) 
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Substitution  of  (8-10)  in  (8-6)  yields 


N  _  l 

p(yj»  y2)  -  C2w  (n-i).T1  ^  +  y^  exp('Vyi  +  yf)  for  a11  yr  y2- 

Special  cases  for  N  equal  to  1  and  2  are  respectively 


and 


p(yr  y2) 


p(yj.  y2)  -  (2*) 


(6-12) 


(B-13) 
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APPENDIX  C.  A  TEST  FOR  HilTENESS  OF  A  SEQUENCE 


r  i  K— 1 

Suppose  data  points  fxkj  are  available.  We  can  form  a  correlation 
estimate  at  delay  n  according  to* 


xk  xk-n 


for  all  n 


(C-l) 


where  summations  without  limits  are  over  the  range  of  nonzero  summands. 

A  - 

is  nonzero  only  for  Jn|  <  K.  If  process  jxn)  were  white,  we  would 
expect  to  have 


Rn  <<  Rq  for  all  n  +  0 


( C— 2 ) 


Therefore  a  measure  of  nonwhiteness  is  afforded  by  the  ratio  E/^,  where  error 


measure  E  is  defined  as  the  sum  of  squares  for  n  +  0, 


E  - 


(C-3) 


However,  (C-3)  is  very  time-consuming  to  calculate  via  (C-l),  because  of  all 
the  multiplications  and  additions  required.  A  much  more  practical  evaluation 
of  (C-3)  is  afforded  by  the  following  procedure  (the  derivation  is  presented 
later  in  this  appendix). 


Define  an  M-point  OFT  of  the  K  data  points: 


K-l 

X_  =  xt  exp(-i2wkm/M)  for  0  <  m  <  M-l  .  (C-4) 

m  k=U  ~  “ 


Then  it  follows  that  if  M  >  2K-1, 


*  The  following  mathematical  development  is  very  similar  to  that  in  [ 7 J . 

1 
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and  our  whiteness  measure  can  be  expressed  as 


A  threshold  test  of  (C-6)  is  equivalent  to 


( C— 6 ) 


non-white 

*  Threshold  .  (C-7) 

white 

The  distribution  of  Q  could  be  computed  from  white-noise  simulations,  and  a 
threshold  value  selected  for  prescribed  error  probability.  By  Schwarz's 
inequality,  Q  2  1.  with  equality  realized  if  and  only  if  j XmJ ^  is  constant 
for  all  m. 

Evaluation  of  Q  in  (C-7)  requires  one  M-point  DFT  of  the  data  (xj^-1, 
where  we  must  have 

M  >  2K  -  1  .  (C-8) 

The  subsequent  calculations  in  (C-7)  are  quickly  conducted. 


An  alternative  interpretation  of  error  measure  E  in  (C-3)  is  very 
illuminating  and  lends  additional  credence  to  (C-7)  as  an  appropriate 
statistic.  If  we  define  spectral  estimate 

6m  =  ^  Rn  exp(-i2*mn/M)  for  all  m  ,  (C-y) 
n 


then  we  find 


for  0  <  m  <  M-l 


(C-1U) 
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and  the  error  measure  E  in  (C— 3 )  becomes 


E 


(C-ll) 


But  notice  that  if  we  define  the  sample  mean  of  the  set  of  spectral 
M-l 


estimates 


as 


u  r 


,  M-l 

h  2 

nwO 


m 


(C-12) 


then  the  sample  variance  becomes 


M-l 

2  1  K  .2 


i  'y  g  2 


(\  ^  * 
■  (j!  ^  6« 


m=0 


(C-13) 


That  is,  error  measure  E  is  equal  to  the  sample  variance  of  the  set  of 
spectral  estimates.  This  latter  quantity  is  a  very  natural  measure  for 
deducing  whether  a  sequence  is  white,  since  would  be  expected  to  be 
smaller  for  a  truly  white  process. 


This  also  suggests  that  a  meaningful  measure,  for  determining  whether  a 
sequence  is  white  over  a  limited  band  of  the  zero-to-Nyquist  range,  is  the 
sample  variance  of  the  spectral  estimates  over  that  particular  band  in 
question. 


K  1 

For  a  real  sequence  fx. }  ”  ,  the  sums  in  (C-7)  need  only  oe  conducted  over 

K,o 


half  of  the  range,  by  virtue  of  the  conjugate  symmetry: 


for  1  <_  m  <_  M-l  if  [xj  real 


(C-14) 
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Thus  then  (for  M  even) 


ixi 


if  |xk]  real.  (C-15) 


DERIVATIONS 


Define  spectral  estimate 


■  ^>f*n  exp(-i2*nm/M)  for  a 


11  m  . 


(C-lb) 


The  superscript  specifically  indicates  the  period  in  tne  subscript  variable. 
Substituting  (C— 1 )  in  (C-16),  there  follows 


2iM)  -  ^exp(-i2imm/M)  x^ 


J  2xk  exp(-i2irkm/M)  j^.  *k_n  exp(-i2*(k-n)m/M)J  * 


I  (M) 
K  m 


( C  — 1 7 ) 


where 


^  x  exp(-i2irkm/rl)  for  all 


(C-18) 


Now  the  inverse  DFT  of 


HM>]  > 


5iM)  exp(12*n/i1) 


...  +  Rn_w  ♦  Rn  *  ^n+M  +  •••  (c~ly) 


for  all  n. 


which  is  an  infinitely-aliased  version  of  [Rn]-  However,  if 


M  >  2K-1 , 


(C-20) 
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there  is  no  overlap  of  terms  in  (C-19).  Henceforth  we  presume  (C-20)  to  be 
true;  that  is,  the  size  of  the  transform  ( C  — 1 8)  must  be  at  least  twice  as 
large  as  the  number  of  data  points.  Then  there  follows  from  (C-19), 


A 


exp  (i2*nm/M) 


for  jn|  <  K 


(C— 2 1) 


Therefore 


n:one  period 
of  length  M 


g(M)  g(M) 
m  p 


exp(i2ir(m-p)n/M) 


Then  from  (C— 3)  and  (C-19), 


E 


(C-22) 


(C-23) 


Although  (C-18)  is  defined  for  ail  relative  sizes  of  M  and  K,  the  relation 
(C-23)  holds  only  if  (C-20)  holds. 
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ABSTRACT 

A  simple  technique  for  determining  the  steepest  descent  contours  of 
Integrands  Is  explained  and  programmed  in  BASIC,  and  then  applied 
to  a  variety  of  examples,  some  of  which  are  very  difficult  to  ascertain 
analytically,  including  one  with  an  infinite  number  of  saddle  points. 
The  amount  of  programming  is  minimal,  requiring  numerical 
evaluation  only  of  the  derivative  of  an  analytic  function. 
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Introduction 


The  evaluation  of  contour  integrals  with  analytic  integrands  is  often  ac¬ 
complished  by  moving  the  contour  to  an  equivalent  one,  such  that  the  integrand  is 
better  behaved  or  can  be  approximated  more  easily.  In  particular,  movement  of  the 
original  contour  to  one  that  takes  advantage  of  saddle  points  of  the  integrand  or 
paths  of  descent  or  steepest  descent  is  a  very  fruitful  procedure.  It  is  presumed  that 
the  reader  is  familiar  with  this  technique;  see  references  1-3,  for  example.  However, 
one  of  the  difficulties  of  this  procedure  is  determining  the  locations  of  the  steepest 
descent  contours  (reference  1,  p.  263).  For  complicated  integrands,  especially  those 
involving  branches  of  multivalued  functions,  exact  determination  of  steepest 
descent  contours  is  virtually  impossible  analytically,  and  recourse  to  some  type  of 
computer  aid  is  recommended.  The  procedure  given  here  does  not  require  solution 
of  nonlinear  equations,  but  does  give  a  very  good  indication  of  steepest  descents 
with  a  minimum  of  analytical  and  programming  effort. 


TR6433 


Explanation  of  Technique 

Suppose  we  wish  to  evaluate  the  contour  integral 

Jdz  g(z)  exp(Xw(z) )  ,  m 

C 

where  g(z)  and  w(z)  are  analytic  functions  of  z,  except  for  isolated  singularities  such 
as  poles,  essential  singularities,  and  branch  points;  and  C  is  a  contour  (finite  or 
infinite)  in  the  complex  z-plane.  The  saddle  points  of  the  exponential  in  (1)  occur 
where  (reference  1,  p.  258) 


w'(za)  •  0  .  (2) 

For  k  real,  the  standard  method  of  determining  the  paths  of  steepest  descent  out  of  a 
saddle  point  is  to  keep  the  imaginary  part  of  w(z)  constant  and  equal  to  its  value  at 
zs  (reference  1,  p.  255).  This  generally  leads  to  difficult  transcendental  equations 
that  must  be  analytically  investigated  approximately  or  solved  numerically. 

An  alternative  procedure  for  finding  the  steepest  descent  directions  at  any  point 
in  the  z-plane  is  as  follows:  Let 

2  *  x  +  iy  , 

....  (3) 

w(x  +  iy)  =  u  +  xv 

Then  the  magnitude  of  the  exponential  in  (1)  is  exp(Au),  and  its  direction  of  steepest 
descent  at  x,  y  is  proportional  to  the  negative  of  the  gradient  (reference  1 ,  p.  254): 

V  exp(Xu)  =  X  exp(Xu)  Vu  ,  (4) 


where 


Vu 


Ju  -  3u  - 
—  a  +  77—  a 

3x  x  3y  y 


(5) 


and"?x  and  a^  are  unit  vectors  in  the  positive  x-  and  y-directions,  respectively.  The 
explicit  evaluation  of  (5)  requires  that  one  analytically  evaluate  u  =  Re{w(x  +  iy)} 
and  then  analytically  derive  3u/3x  and  du/dy.  This  can  be  tedious  and  is  liable  to 
human  error. 


An  alternative  simpler  procedure  is  possible:  By  the  Cauchy-Riemann  conditions 
applied  to  a  function  analytic  at  z,  the  derivative 


w  (z) 


3u  .  3v 

3x  +  1  3x 


jv  3u 

~>y  ’  1  ~>y 


That  is,  we  can  express  the  desired  partial  derivatives  as 


(6) 


’U  n  '  (  \  ->U 

—  =  Re i w  (:) },  —  = 

•  x  .> 


-  Im{w' (:)  ; 


(7) 


TR  6433 


Then  the  negative  of  the  gradient  in  (3)  can  be  written  as 

-Vu  *  -Re{w'(z)}  a  +  Im{w'(z)}  a  .  (8) 

x  y 

So  if  we  evaluate  w  '(z),  the  direction  of  steepest  descent  at  any  point  z  has 
components  in  the  x,  y  directions  proportional  to 

-Re{w'(z) },  Im{w'(z)}  (9) 

for  w  '(z)  +  0.  The  only  analytical  calculation  necessary  is  that  of  derivative  w  '(z),  a 
task  generally  easily  accomplished,  and  indeed  necessary  for  evaluation  of  saddle 
point  locations  anyway.  A  computer  program  can  then  be  written  to  numerically 
evaluate  w  '(z)  in  terms  of  its  components  (9)  at  all  points  of  interest  in  the  z-plane. 
A  program  for  this  procedure  is  given  in  appendix  A,  along  with  the  specific 
examples  displayed  later  in  this  report. 

Presentation  of  this  steepest  descent  information  for  human  interpretation  is 
accomplished  here  by  drawing  a  short  standard-size  line  through  each  point 
z  =  x  +  iy,  centered  on  the  point  and  with  an  arrowhead  pointing  in  the  direction 
of  steepest  descent.  The  magnitude  of  the  rate  of  steepest  descent  is  discarded;  only 
the  direction  is  preserved.  How  effective  this  procedure  is  will  be  demonstrated  by 
the  following  examples. 


3 


TR  6433 


Examples 


Airy  Function 

The  Airy  function,  Ai,  is  proportional  to  ( 1 )  when  g(z)  =  1 , 

w(z)  =  z  -  yZ3  ,  00) 

and  C  is  an  infinite  contour  starting  anywhere  in  the  angular  range  -5rr/6  <  arg  (z) 
<-n/2  and  ending  in  n/2  <  arg(z)  <5rr/6;  see  reference  1 ,  pp.  52  and  266.  Then 

w;(z)  =  1  -  z"  .  (11) 

A  computer  need  only  evaluate  the  complex  product  z*z  and  subtract  it  from  1  in 
order  for  (11)  to  be  used  in  (9).  An  example  of  this  procedure  is  given  in  figure  1. 
The  arrows  clearly  indicate  the  steepest  descent  paths  from  any  point  in  the  z-plane. 
The  two  solutions  of  (1 1)  equal  to  zero,  namely,  saddle  points  zs  =  ±1 ,  have  arrows 
pointing  both  inward  and  outward  at  these  points,  reflecting  the  very  nature  of  a 
saddle  point.  Movement  of  the  original  contour  C  to  the  steepest  descent  contours 
(solid  lines)  out  of  the  saddle  point  at  zs  =  -1  is  easily  accomplished;  no 
singularities  of  the  integrand  of  (1)  are  crossed  in  the  movement  process. 

The  Airy  function  for  complex  argument  (reference  1,  section  7.3)  has,  more 
generally, 

w(z)  =  zexp(it))-  yz3  .  (12) 

Figure  1  corresponded  to  8  =  0.  For  0  #  0,  the  determination  of  steepest  descent 
paths  is  analytically  difficult  (ibid.).  However,  since 

w'(z)  =  exp(i9)  -  z“  ,  (13) 

computer  evaluation  of  (13)  and  (9)  is  trivial.  The  steepest  descent  directions  for 
Q  =  3tt/4,  for  example,  are  depicted  in  figure  2.  The  solid  lines  in  the  neighborhood 
of  the  saddle  points  were  hand-drawn  upon  observation  of  the  descent  arrows.  The 
interpretation  of  figure  2  is  much  easier  than  its  counterpart  in  reference  I,  figure 
7.3.3.  Also  the  determination  of  an  equivalent  contour  to  C  is  easily  achieved  by 
reference  to  figure  2.  First  let  the  new  contour  come  from  °°  exp(-i2n/3)  along  a 
steepest  ascent  to  the  saddle  point  at  zs  =  exp(-i5n/8).  Then  let  it  continue  in  a 
northeasterly  direction  along  the  steepest  descent  direction  out  of  this  saddle  point 
to  the  y-axis.  Next  proceed  due  north  to  a  point  near  z  =  i  and  then  follow  a 
steepest  descent  path  out  to  »exp(i2n/3).  The  vertical  portion  of  this  new  contour  is 
not  a  path  of  steepest  descent,  but  it  is  obviously  a  path  of  descent  because  the 
projections  of  the  arrows  on  this  vertical  section  all  point  in  the  upward  direction  of 
travel.  This  discussion  also  points  out  that  the  other  saddle  point  at  zs  =  exp  (i3n/8) 
does  not  enter  into  the  asymptotic  development  of  Ai,  at  least  for  this  value  of  6. 

An  alternative  equivalent  contour  to  C  is  the  pair  of  steepest  descent  contours 
passing  through  the  saddle  points  and  connecting  °°exp(-i2n/3)  to  +  00  and  +  00  to 
°°exp  (i2n/3),  respectively;  see  figure  2.  Observe  that  the  movement  of  C  to  this  new 
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pair  of  paths  is  not  an  approximation;  it  is  an  alternative  exact  representation  of  the 
original  integral.  If  one  now  approximates  these  two  path  integrals  by  their  con¬ 
tributions  near  their  peaks  at  the  saddle  points,  the  saddle  point  at  zs  =  exp(i3n/8) 
will  yield  exponentially  small  contributions  relative  to  that  at  zs  =  exp(-i5n/8). 
This  can  be  seen  from  figure  2  by  drawing  a  straight  line  between  the  two  saddle 
points;  all  projections  of  arrows  along  this  line  point  at  the  upper-right  saddle  point, 
meaning  that  the  value  of  |exp(Aw(z))|  is  smaller  there. 


Figure  I.  Steepest  Descent  Directions  for  Airy  Function.  0  =  0 
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When  we  continue  on  to  the  case  6  =  n  in  (12),  the  character  of  steepest  descents 
is  as  depicted  in  figure  3.  Now  the  straight  line  connecting  the  two  saddle  points  has 
all  arrows  perpendicular  to  it;  thus  this  vertical  line  is  a  contour  of  constant 
|exp(Aw(z))|.  This  means  that  both  saddle  points  contribute  equally  to  the  value  of 
integral  (1).  Again  the  pair  of  steepest  descent  contours  through  the  saddle  points 
(mentioned  in  the  above  paragraph)  represent  exactly  the  original  integral;  one 
could  evaluate  the  original  integral  exactly  by  adding  the  total  contributions  of  both 
of  these  paths,  or  an  approximation  can  be  achieved  by  computing  the  integrand 
near  its  peaks  at  the  saddle  points. 
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Figure  3.  Steepest  Descent  Directions  for  Air>  Function,  9  =  n 
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Hankel  Function 

The  Hankel  function  takes  the  form  (reference  1, 7.2.23) 

w(z)  =  i[cos(z)  +  3(z  -  ^-)]  ,  (14) 

where  /)  is  a  constant.  There  follows  immediately 

w'(z)  =  i[-  sin(z)  +  3]  .  (15) 

Computer  evaluation  of  (15)  requires  only  a  trigonometric  sin  of  a  complex  number, 
followed  by  subtraction  and  multiplication.  Sample  descent  direction  plots  for  ft  = 
0.5,  1,  and  1.5  are  given  in  figures  4-6,  respectively.  (It  is  informative  to  compare 
these  figures  with  figures  7.2.1 -7.2.4  in  reference  1.)  Movement  to  equivalent 
contours  is  obvious  from  figures  4-6.  Since  (15)  has  period  2tt  in  x,  only  a  2n  strip 
has  been  plotted  in  figures  4-6.  The  character  of  the  steepest  descents  in  figure  5  for 
p  =  1  is  different,  in  that  the  saddle  points  have  coalesced;  however,  there  is  no 
difficulty  ascertaining  from  the  plots  what  new  contour  to  adopt. 

Klein-Gordon  Equation 

This  example  is  complicated  by  the  presence  of  branch  lines  in  the  exponent 
function  w(z).  Specifically  we  have  (reference  1 , 7.5.9) 

w(z)  =  i [e ( z 2  -  1)*/2  -  z]  ,  (16) 

where  the  branch  of  the  square  root  is  taken  as  positive  real  for  z  =  x  >  1 ,  and  with 
branch  lines  extending  vertically  downward  from  the  branch  points  at  z  =  ±1.  The 
derivative  of  (16)  is 


where  the  same  square  root  branch  must  be  taken  as  in  (16). 

If  one  has  available  a  computer  program  that  evaluates  the  principal  square  root 
of  a  complex  number,  denoted  here  by  z  ,  it  can  be  used  to  evaluate  (17)  in  the 
following  manner.  Observe  first  that  the  branch  line  of  principal  square  root  z 
occurs  where  z  =  -p  for  p  >  0;  i.e.,  p  can  take  on  all  nonnegative  real  values.  So 
consider  the  representation 

(=2  -  l)‘/2  =  i(-ii  *  >)l/2  (-iz  -  i)1/:  .  (18) 

For  z  real,  positive,  and  large,  the  right-hand  side  of  (18)  approaches 
i[esp(-in/4)z  ]  [e\p(-in/4)z  ]  =  z,  as  desired.  Furthermore,  the  two  branch  lines 
of  ,18)  occur  where  the  arguments  of  the  two  principal  square  roots  have  values 

-i  z  ♦.  i  =  -p  for  p  >  0  ; 

i.e.,  -  =_♦!  -  ip  for  p  0  (19) 
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Figure  5.  Steepest  Descent  Directions  fur  Hankel  Function.  ft=  I 


10 


\  \  V  ^ 
\  \  \  ^ 


\  \  \  V 

\  \  \  V 

\  \  \  V 

\  \  \  V 


\ 

\ 

\ 

\ 

\ 

\ 

\ 

\ 

\ 

\ 

\ 

t 

t 

\ 

\ 

^SADDLE- 


\  \  \  \  \  t  t  t  t 


t  f  /  ' 
If/' 
f  t  /  ' 
If/' 


^  'X  \ 


If/' 

t  /  '  ' 


^  \  \  l  /  ' 

^  \  \  /  /  ✓ 


TR  6433 


These  are  vertically  downward  from  z  =  ±1,  as  desired.  Thus  (17)  can  be  easily 
evaluated  by  taking  the  two  complex  principal  square  roots  indicated  in  (18)  and 
performing  multiplication,  division,  and  addition  of  complex  numbers.  The  specific 
coding  is  illustrated  in  appendix  A. 

The  pole  of  g(z)  at  z  =  v0  for  this  example  (reference  1,  7.5.8)  has  no  effect  on 
the  steepest  descent  contours  of  w(z).  The  steepest  descent  directions  for  6  -  0.8 
are  depicted  in  figure  7.  There  are  saddle  points  at  z$  =  ±5/3,  and  the  steepest 
descent  contours  go  vertically  downward  eventually.  The  steepest  descent  directions 
near  the  branch  lines  emanate  from  the  branch  lines  themselves,  but  these  branch 
lines  have  no  effect  on  the  steepest  descent  contours  through  the  saddle  points. 
However,  if  the  branch  line  emanating  from  the  branch  point  at  z  =  1  had  been 
taken  at  angle  -n/6,  for  example,  it  would  have  interfered  with  the  steepest  descent 
contours  in  the  4th  quadrant  of  the  z-plane.  Such  a  choice  of  branch  for  the  square 
root  in  (16)  and  (17)  is  undesirable  and  should  be  avoided,  as  was  done  in  figure  7. 


Function  with  Essential  Singularity 


This  integrand  is  characterized  by  (reference  4) 


exp(w(z) ) 


exp 


(tH 


(20) 


which  function  has  a  zero  at  z  -  0  and  an  essential  singularity  at  z  =  l.Then 

w(z)  =  In (z) -  £-K—  , 


w'(z) 


1  K 


(21) 


The  steepest  descent  contours  for  (21)  are  depicted  in  figure  8  for  K  =  3.  The 
essential  singularity  generates  a  “dipole  effect”  about  z  =  1,  i.e.,  0  at  z  =  1  +  ,  and 
00  at  z  =  1-,  for  K  >  0.  The  zero  of  (20)  at  z  =  0  manifests  itself  as  a  point  toward 
which  all  the  arrows  point,  since  zero  is  the  smallest  magnitude  that  any  complex 
function  can  take  on.  The  saddle  points  (roots  of  (21))  occur  at 


z 

s 


exp(+.i6),  where  9  =  arc  cos 


for  0  <  K  <  4 


(22) 


Utilizing  the  information  in  figure  8,  we  find  it  relatively  easy  to  decide  what  the 
effect  of  moving  an  original  contour  around  in  the  z-plane  will  do  to  integral  (1). 
Movement  across  the  essential  singularity  at  z  =  1  will  necessitate  consideration  of 
the  residue  at  this  point.  The  zero  and  saddle  points  of  (20)  are  not  points  of 
singularity. 


Cubic  Function 


This  example  comes  from  reference  3,  pp.  296-302;  it  is  characterized  b\ 
g(z)  =  '/:  and  (ibid.,  upper  line  of  6.6.25) 
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Figure  7.  Steepest  Descent  Directions  for  Klem-(iordnn  Kgualion, 
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Figures.  Steepest  Descent  Directions  fur  exp  (w(z))  =  zexp(“j^,  K  =  3 

Also  now  the  integral  is  a  finite  one,  from  z  =  -Itoz  =  +  1 .  This  example  exhibits 
a  Stokes  phenomenon  at  certain  values  of  a,  where  we  have  represented  A  =  |A|eio. 
We  find 

w’(z)  =  -e*a  ^8z  -  i.5  +  i5:“)  ,  (24) 

which  has  zeros  (saddle  points)  at  z5  =  iandi5/3. 

A  plot  of  steepest  descent  directions  for  a  =  0  (positive  real  A)  is  given  in  figure  9. 
It  indicates  that  the  steepest  descent  contours  out  of  the  limits  at  z  =  - 1 ,  +1  tend  to 
°°exp(-i5n/6)  and  «exp(-in/6),  respectively.  But  these  two  valleys  at  °°  can  be 
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Figure  9.  Steepest  Descent  Directions  for  Cubic,  a  =  0 


joined  by  the  saddle  point  contribution  through  the  point  zs  =  i.  Thus  the  integral 
over  (-1,  1)  is  exactly  equal  to  the  sum  of  these  three  steepest  descent  contours.  The 
saddle  point  at  z5  =  i5/3  need  not  be  considered.  The  dominant  contribution  is 
obviously  that  at  the  saddle  point  zs  =  i,  as  may  be  seen  by  the  arrow  directions. 

For  a  =  5n/12,  figure  10  indicates  a  similar  behavior.  Since  the  steepest  descent 
contours  out  of  -1,  +1  tends  to  -175°  and  -55°, respectively,  the  saddle  point  at 
zs  =  i  must  again  be  used  to  join  them.  The  dominant  contribution  is  seen  to  be  due 
toz  =  -1,  by  making  use  of  the  arrows  of  descent  in  this  figure. 

For  o  =  3n/4,  however,  we  see  from  figure  1 1  that  both  of  the  steepest  descent 
contours  out  of  -1,  +1  tends  to  °°exp(-i5n/12).  Now  there  is  no  need  to  employ 
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either  of  the  saddle  points.  The  integral  over  (-1,1)  is  given  exactly  by  the  sum  of  the 
two  steepest  descent  contributions.  The  dominant  contribution  is  again  due  to 
z  =  -1,  since  we  have  to  descend  from  z  =  -1  to  near  z  =  0  to  reach  magnitude 
values  comparable  to  those  at  z  -  + 1 . 


Figure  10.  Steepest  Descent  Directions  for  Cubic,  a  =  5n/l2 
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Figure  II.  Steepest  Descent  Directions  for  Cubic,  a  =  3n.  4 


Gaussian  Exponent 

The  characteristic  function  of  a  particular  type  of  impulsive  noise  is  given  in 
reference  5,  equation  (5),  in  the  form 

■■  .  m 

f(')  =  cxp(-A)  £  —  exp 
m=(>  • 

for  a  purely  Poisson  process  (no  Gaussian  background).  This  summation  can  be 
evaluated  in  the  closed  form 


.  mB~ 
1  o 


(25) 
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.  7  2 

f(C)  =  exp  I A  exp  l-y  -y  5 


-  A 


(26) 


The  corresponding  probability  density  function  of  this  impulsive  noise  is  given  by 
the  Fourier  transform 


(27) 

By  expanding  f(4)  in  (26)  in  a  power  series,  we  note  that  the  mean  square  value  of 
the  Poisson  process  is  readily  found  to  be  ABz/2.  Since  we  will  be  interested  in 
values  for  the  dimensionless  parameter  A  of  the  order  of  1  (e.g.,  A  =  0.35  in 
reference^  5,  figure  3),  we  will  normalize  our  random  variable  according  to 
t  =  v/(Bz/2 )‘/l  =  v/o0.  Also,  as,  4  -*  ±«,  f(4)  in  (26)  tends  to  nonzero  value 
exp(-A).  Adding  and  subtracting  this  quantity  and  letting  |  =  z/oQ  enables  (27)  to 
be  expressed  as 

p(v)  *  exp(-A)  6(v) 


p(v) 


« J_7 

2  it  J 


d£  exp  I -i£v  +  A  exp 


\ 


4-oo 

+  /  dz  exp(-itz)[exp(A  exp(-z2/2))  -  l]  , 

o  (28) 

Although  there  is  no  obvious  parameter  A  in  this  form,  it  is  shown  in  appendix  B 
that  we  can  still  use  steepest  descent  procedures  on  the  logarithm  of  the  integrand  of 
(28);  i.e.,  here  we  have 

w(z)  =  -itz  +  ln[exp(a(z))  -  1]  ,  (29) 

where  we  have  defined 

? 

a(z)  *  A  exp(-z  /2)  .  (30) 

Observe  that  as  z  -*■  °°  with  arg(z)  in  the  two  sectors  within  n/4  of  the  positive-real 
or  negative-real  axes,  a(z)  becomes  very  small  and 

w(z)  ~  -itz  +  ln[a(z)]  =  -itz  +  In  A  -  z “/2  ;  (31) 

this  means  that  Re  w(z)  -*  in  these  two  sectors  of  the  z-plane,  and,  therefore,  the 
integrand  of  (28)  tends  to  zero  in  these  sectors  as  z  -* 

The  integrand  of  the  integral  of  interest  in  (28)  is  zero  in  the  finite  z-plane  only 
when 

h(zQ)  5  exp^Ae'Z°^2j  -1=0, 

Ae  “0/  “  =  i2nn  for  n  f  0 
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-  -j-  3  ln^i  +  i2irm  *  +  ij  sgn(n)  +  i2irm  , 

zq  =  +,i  ^2  In  (  ^’T|,n  ^  )  +  iit(4m  +  sgn(n)) 

where  A  >  0,  the  square  root  is  the  principal  branch,  and  n  and  in  are  arbitrary 
integers  (negative,  zero,  or  positive),  except  that  n  #  0.  These  zeros  of  h(z)  are 
important  because  they  will  be  locations  toward  which  all  the  steepest  descent 
arrows  must  point  in  their  neighborhoods,  since  zero  is  the  smallest  magnitude  that 
a  complex  function  can  take  on.  These  zero  locations,  of  which  there  are  an  infinite 
number,  depend  only  on  A,  and  not  on  normalized  variable  t  =  v/oD  in  expression 
(28)  for  the  probability  density  p(v).  The  locations  of  the  zeros  of  h(z)  in  the  third 
quadrant  closest  to  the  origin  are  depicted  in  figure  12  for  A  =  0.35.  There  is 
symmetry  in  the  other  quadrants  since  (from  the  first  line  of  (32)) 


h(-z)  =  h(z)  ,  h(z*)  =  h* (z)  .  (33) 

For  purposes  of  evaluating  steepest  descent  directions  and  saddle  point  locations, 
•  we  note  that  a(z)  in  (30)  has  the  property 


a' (z)  =  -za(z) 


(34) 


and  so  (29)  yields 


w  (z)  =  -it 


z  a(z) 

1  -  exp(-a(z)) 


(35) 


Thus  the  saddle  points,  zs,  of  which  there  are  an  infinite  number,  are  solutions  of 


*s  a(zs} 

1  -  expf-af:^))  "  **  ’  (36) 

and  obviously  depend  on  both  A  and  t.  The  most  important  saddle  point  is  at 
zs  =  -ip,  (i  positive  real,  where  (using  (30)), 

■A  exp(i~/2)  _ 

*7  """  ~  ^  • 

1  -  cxp(-A  exp(r“/2))  (37) 

The  steepest  descent  directions  for  A  =  0.35,  t  =  2  are  depicted  for  (35)  in  figure 
13.  The  saddle  point  at  zs  =  -ip  =  -il  .341  satisfies  (37);  there  are  five  other  saddle 
points  indicated  by  X  in  the  figure,  in  addition  to  the  zeros  carried  over  from  figure 
12.  The  steepest  descent  contour  out  of  the  saddle  point  zs  =  -i  1 .34 1  is  drawn  as  a 
solid  line;  it  is  asymptotic  to  y  =  -t  as  z  -*  «  with  arg(z)  within  n/4  of  the  positive- 
and  negative-real  axes.  Movement  of  the  original  contour  from  the  real  axis  in  (28) 
to  the  steepest  descent  contour  is  easily  justified. 
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Figure  12.  Zero  locations  for  A  =  0.3S 
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To  deduce  the  asymptotic  nature  of  the  steepest  descent  contours,  recall  (31);  thus 
the  imaginary  part  of  w  is 

v ~ -tx  -  xy  =  -x(y  ♦  t)  as  z  -*•  «  (38) 

in  the  sectors  under  consideration.  But,  since  from  (29), 

w(-i£)  =  -fit  +  ln[exp(A  expvo5“/2))  -  l]  (39) 

is  real,  we  require  (38)  to  approach  0  as  x  -*  ±«.  This  requires  that  y  -*  -t.  as 
claimed. 

We  observe  from  figure  13  that  no  use  is  made  of  the  saddle  point  at  (-1.71, 
-2.38),  nor  of  the  infinite  number  of  other  saddle  points.  We  also  observe  that  the 
descent  directions  in  the  neighborhood  of  the  closely  spaced  zeros  and  saddle  points 
near  the  bottom  of  the  figure  is  very  detailed  and  complicated;  however,  none  of 
that  information  is  needed. 

When  t  is  increased  to  5,  figure  14  applies.  Now  the  steepest  descent  contour  out 
of  the  saddle  point  at  zs  =  -i/J  =  — il  .945  heads  into  the  zero  at  (-0.632,  -2.485). 
How  to  connect  from  this  latter  point  to  z  =  -»  is  not  clear.  Instead,  we  consider  a 
horizontal  descent  contour  out  of  zs  =  -i  1.945  until  we  get  in  the  neighborhood  of 
(-2,  -2),  and  then  we  resume  a  steepest  descent  contour  heading  towaid 
y  =  -t  =  -5.  The  major  contribution  to  this  descent  contour  is  given  by  the 
neighborhood  of  the  saddle  point  at  x  =  0;  the  descent  and  steepest  descent  con¬ 
tours  are  tangent  at  this  saddle  point. 

The  aid  afforded  by  the  steepest  descent  directions  depicted  in  figures  13  and  14  is 
extremely  worthwhile,  since  the  exponential  in  (29)  and  (35)  makes  an  analytical 
approach  very  difficult.  The  ability  to  discard  or  avoid  certain  regions  of  the  z-plane 
in  determining  an  appropriate  descent  contour  is  rather  obvious  from  the  figures 
when  coupled  with  basic  information  about  the  integrand,  like  the  asymptotic 
behavior  of  the  steepest  descent  contour. 

The  above  results  yield  exact  values  for  the  original  integral,  since  we  have  simply 
determined  contours  equivalent  to  those  originally  specified.  Now  we  will  derive  an 
asymptotic  expansion  for  the  probability  density  in  (28)  for  large  t  =  v/o0,  i.e. ,  at 
values  v  much  larger  than  the  standard  deviation  of  the  Poisson  process. 

To  do  this,  we  need 


where  a  =  a(z)  is  defined  in  (30).  Then  we  find 

w"(zs)  =  t2  *  (l  *  zs  -  zs  a(zs})  (41) 

s 

and,  in  particular, 

w"(-i6)  =  t2  -  +  32  ♦  B2  A  exp(32/2))  ,  (42) 
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Figure  14.  Descent  Contours  for  A  =  0.35, 1 
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for  the  saddle  point  at  zs  “  —i/3.  Also  w(— i/3)  is  given  by  (39).  Then  we  have  the 
approximation  (reference  1 ,  chapter  7) 


p(v)~ 


exp(-A  -  gt) 
2ira 

o 


»  " 

exp^A  exp(B2/2)^-  1 

(  2tt 

(j^'C-iB)  | 

1/2 


as  t 


(43) 


One  drawback  with  this  solution  is  that  ft  depends  on  t  through  the  solution  of 
transcendental  equation  (37).  For  large  t,  we  have,  to  first  order, 

g  as  [2  ln(t/A)]1/2  =  L1/2  .  (44) 


For  development  of  additional  terms  and  the  general  philosophy  of  solution  of 
these  types  of  problems,  see  reference  6,  pp.  11-16  and  83-84.  We  find,  more 
generally, 

B2  »  L  -  j-—-  In  (L)  .  (45) 

Even  so,  substitution  into  (43)  yields  a  very  complicated  expression  for  the 
probability  density  function  unless  t  is  excessively  large.  It  can  be  seen  that  (43) 
decays  slightly  faster  than  an  exponential  for  large  t. 
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Summary 

A  technique  for  simple  determination  of  the  steepest  descent  direction  at  any 
point  in  the  complex  plane  has  been  presented  and  illustrated  with  numerous 
examples.  The  movement  of  the  original  contour  to  an  equivalent  descent  or 
steepest  descent  contour  is  an  exact  representation  and  can  be  deduced  fairly  easily 
from  the  descent  information.  At  this  point,  two  alternatives  are  available,  either 
exact  numerical  evaluation  of  the  integral  or  an  approximation  such  as  Laplace’s 
method.  Very  difficult  integrands  can  be  handled  very  effectively  via  this  approach. 
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Appendix  A 

Computer  Programs 

The  major  part  of  the  calculations  required  for  the  six  examples  in  the  main  text  is 
that  of  w'(z).  This  is  accomplished  in  the  Subroutine  Wderivative  in  the  main 
program  written  in  BASIC  and  listed  at  the  end  of  this  appendix.  But,  first,  the  six 
subroutines  for  the  examples  given  are  listed.  They  illustrate  how  little  program¬ 
ming  is  actually  needed  to  compute  w  '(z),  provided  that  one  has  already  written 
subroutines  for  the  standard  complex  operations  and  functions  like  multiply, 
divide,  exp(z),  log(z),  square  root,  arg(z),  sin(z),  cos(z),  etc.  These  latter  functions 
are  listed  for  completeness  as  subroutines  at  the  end  of  the  enclosed  program. 

410  SUB  Wder  i  vat  i  ue  <X,  Y,  Rewl ,  I  mwl  >  !  Airy  function 

411  COM  Ct,St  !  co*< theta),  i i  n < t he t a> 

412  CALL  Mul <X, Y, X, Y, A, B> 

420  Rewl»Ct-A 

430  I»ul«St-8 
440  SUBEND 


410  SUB  Wder  i  uat  l  *»e<X,  Y,  Rewl ,  Imwl  >  1  Hank*'  f  ur.c  *  .on 

411  Beta*. 5 

412  CALL  S i  n<X,  Y,  A,  B ) 

420  Rewl*B 

430  Imwl«Beta-A 
440  SUBEND 


410  SUB  Wder  i  "at »  v*  <X,  Y ,  Rew  1 ,  I  mwl )  1  >. '  t  i  n -  Z  :  r  dor 

411  Theta*. 8 

412  CALL  Sqr <V, -X+ 1 , A, B> 

413  CALL  Sqr<Y,-x-l,C,D) 

414  CALL  Mu  1 <  A, B , C , D , E , F  > 

413  CALL  D  i  <■•<  The »  a*X ,  Thet  a*Y ,  E  ,  F ,  C ,  H  1 
420  Rewl*G 
430  Imwl»H-l 
440  SUBEND 


410  SUB  tide  r  l  v  at  i  •<«  <  X ,  Y ,  Re'.i  1  ,  I  rn  ■  1  1  Eiien*  i  tl  j  n  j  j  I  ,  i»  • 

411  K=3 

412  CALL  Mul < X-l ,  Y, X-l , Y, A, 

413  CALL  Di-<K,0,  A,  B,C.  D  • 

4  14  CALL  D l  1 , 0 .  X.  Y,  E,  F  ' 

420  F'«  !•'  1  *C  »E 

4  3  0  I  in  i.)  1  =  D  +  F 

440  SUBEND 
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410  SUB  Wd«r i v*t i v*<X, Y , R*wl , I mwl >  1  cubic  function 

411  COM  C»,  $*  !  coi<*1ph*>,  tin<alph*> 

412  CALL  Mul <X, Y,  X, Y, A, B> 

413  CALL  Mul <C», S*,  8*X-3#B, 8*Y-5  +  3*A, C, D> 

420  R»ul»-C 

430  Imwl*-D 
440  SUBEND 


410  SUB  Wd*r  i  vat  i  v*<X,  Y ,  R*wl ,  I hiw  1  >  !  Gauisian  exponent 

411  A-.33 

412  T  »2 

413  CALL  Mul <X, Y, X, Y, T1 , T2> 

414  CALL  Exp<-.S*T1,-.5*T2,T3,T4> 

415  Ar»A*T3 

416  Ai *A*T4 

417  CALL  Exp<-Ar, -Ai , Er, Ei  ) 

418  CALL  Mul <X,Y,Ar,Ai ,T1,T2> 

419  CALL  Div<Tl, T2,l-Er,-Ei,T3,T4> 

420  R*wl— T3 

430  Imwl»-T-T4 

440  SUBEND 
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1  ! 
10 
20 
30 
40 
50 
60 
70 
80 
90 
100 
110 
120 
130 
140 
150 
160 
170 
180 
190 
200 
210 
228 
230 
240 
250 
260 
278 
280 
290 
30O 
310 
320 
330 
340 
350 
360 
370 
380 
390 
400 
418 
420 
430 
440 
450 
460 
4T0 
460 
490 
500 
510 
520 
5  30 
540 


Steepest  Descent  oi*  w'(z);  use  SUB  Wdertvatiue  in  line  410 


LEFT  ABSCISSA 
RIGHT  ABSCISSA 
BOTTOM  ORDINATE 
TOP  ORDINATE 
X  INCREMENT 
Y  INCREMENT 


!  GRID  LINE  SPACING 


XI— 3 
X2-3 
Y 1—3 
Y2*3 
Dx* .  2 
Dy*.  2 

PLOTTER  IS  "GRAPHICS" 

GRAPHICS 

SCALE  X1,X2,Y1,Y2 
LINE  TYPE  3 
GRID  1,1 
LINE  TYPE  1 

F*. 2*SQR<Dx*Dx+Dy*Dy >  i  ARROW 

Q1«1-C0S<PI/12>*.8  *  INFOR- 

Q2*SIN<PI"'12)*.S  !  MATION 

FOR  X*X1  TO  X2  STEP  Dx 
FOR  Y-Yl  TO  Y2  STEP  Dy 
CALL  Uder i vat i vetX, Y, Rewl ,  I mw 1 ) 

CALL  Di rec  t i on < Rewl , Imwl ,  Cos , Si n) 

IF  ABS<Cos)+ABS<Sin)>0  THEN  230 
OUTPUT  0; "SADDLE  POINT  AT  ";X;Y 
GOTO  358 
Tl*F*Cos 
T2-F*Sin 
X**X*T1*Q1 
Xb-T2*Q2 
Y*«Y*T2*Q1 
Yb*Tl#Q2 
MOVE  X-T1.Y-T2 
DRAW  X*T 1 , Y+T2 
MOVE  X**Xb,Y*-Yb 
DRAW  X+T 1 , Y+T2 
DRAW  X*-Xb , Y*+Yb 
PENUP 
NEXT  Y 
NEXT  X 
PAUSE 

DUMP  GRAPHICS 
END 

i 

SUB  Wde r  i  ■> at  i  •»«  < X ,  V ,  Re  w  1  ,  I rt. •.*  1  '• 

Rew  1  *  1  -::*»y*y 

I  1  — 2*X*Y 
SUBEND 


!  w'  Cz> 

!  direction  ot  steepest  descent 


hit''  t'.  n  c  i  • 
w  ■  z  .  *  z  -  z  3  . 
w  1  z  '  =  1  -  z  2 


SUE  Direct i on v  Re  w 1 , I  mu  1 , C os 

T *S0R  <  Pew  1  *Re  w  1  ♦  I  n, m  1  *  I  rnw  1  > 

IF  TO  THEN  510 

Cos  *S l n*0 

GOTO  530 

Cos=-Rewl  'T 

Sin* Imwl  T 

SUBEND 


b  i  n  > 


direct  i  o  n  o  t’ 
steepest  d  e  s  c e n ♦ 
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550  SUB  Mu) <X1,Y1,X2, Y2,fi,B>  1  21*22 

560  A«X1*X2-Y1*Y2 

570  B*X1*Y2+X2#Y1 

58d  SUBEND 

590  ! 

600  SUB  Diu<Xl,Yl,X2,Y2,A,B>  !  21/22 

610  T»X2*X2+Y2*Y2 

620  A*<X1#X2+Y1*Y2>/T 

620  B»<Y1#X2-X1*Y2>/T 

640  SUBEND 

650  ! 

660  SUB  Exp<X,Y,A,B>  !  EXP<2> 

670  T-EXP(X) 

680  A»T*COS< Y) 

690  B«T*SIN<Y> 

700  SUBEND 
710  ! 

720  SUB  Log<X, Y, A, B )  !  PRINCIPAL  L0G<2> 

730  A».5*L0G<X*X*Y*Y> 

740  IF  XO0  THEN  770 
750  B*. 5*PI*SGN<Y) 

760  GOTO  790 
770  B«ATN<Y/X> 

780  IF  X<0  THEN  B«B*PI*<1-2*<Y<0> > 

790  SUBEND 
808  ! 

810  SUE  Sqr^X, Y, A, B >  !  PRINCIPAL  SGft<2> 

320  IF  XO0  THEN  860 
830  A»B«SQR<.5*ABS<Y>> 

340  IF  Y< 0  THEN  B«-B 

850  GOTO  970 

860  F-SQR<SQR<X*X*Y*Y>> 

870  T».5*ATN<Y/X) 

880  A-F*COS<T> 

890  B-F*SIN<T) 

900  IF  X>0  THEN  970 
910  T-A 
920  A«-B 
930  B*T 

940  IF  Y>«0  THEN  970 
950  A*-A 
960  B*-B 
970  SUBEND 
980  ! 

990  SUB  A  r  g  <  X ,  Y ,  A  )  !  PRINC  IPAL  ARC ■  Z  •' 

1000  IF  X*0  THEN  A*.  5*PI*SGN<  Y  "• 

1010  IF  XO0  THEN  A  =  A T N v  Y / X ) 

1O20  IF  X<0  THEN  A*A*PI*< l-2*< Y' 0  ’  ) 

1030  SUBEND 
1040  ! 

1050  SUB  Pc<«r ' X, Y, P, A, B  •  1  PPINCIFAL  PQMEF  1  P 
106O  F  =  EXP <  .  5 » R * L O G  ■,  X * X  +  Y * Y  > 

107O  CALL  Arg » X , Y , T  • 

1 080  A*F*C  OS1.  R*T> 

1090  B  =  F  *  S  I N  '■  R  *  T  ,* 

1100  SUBEND 
1110  ! 
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1120  SUB  Sln<X,Y,A,B>  !  SIN<Z> 

1130  E*EXP< Y) 

1140  ft«.5*SlN<X  >#<£•►  1/E> 

1150  IF  flB$< Y>< . 1  THEN  1180 
1160  S».5#<E-1/E> 

1170  GOTO  1200 
1180  S  =  Y*Y 

1190  S«Y*<120^S*<20+S)>/120 

1200  B*COS<X)*S 
1210  SUBEND 
1220  ! 

1230  SUB  Co*<X,Y,fl,B>  !  COS<,2> 

1240  E»EXP<Y) 

1250  fl*.  5#C0S<X)*<E  +  1"'E> 

1260  IF  flBS<  YK  .  1  THEN  1290 
1278  S-.5*<E-1/'E> 

1280  GOTO  1310 
1290  S«Y#Y 

1300  S-Y*<120*S*<20+S>  >/120 

1310  B«-SIN<X)*S 
1320  SUBEND 
1330  • 

1340  SUB  Sinh<X,Y,fl,B)  !  SINH<2,' 

1350  E«EXP<X) 

1360  B*.5*SIN<Y)*<E*1/E> 

1370  IF  flBSCXX.l  THEN  1400 
1380  S*.5*<E-1/E> 

1390  GOTO  1420 
1400  S«X*X 

1410  S-X*<120+S*<20t-S')^i20 

1420  fl«COS<Y)*S 
1430  SUBEND 
1440  ! 

1450  SUB  Co»h<X,  Y, fi,B)  !  COSH1  2  • 

1460  E-EXPOO 

1470  fl“. 5*C0S< Y E+ 1 /E  > 

1480  IF  fiBS<X>< . 1  THEN  1510 
1490  S«.5*<E-1/E) 

1500  GOTO  1530 
1510  S«X*X 

1520  120  +  S*',20  +  S>  >/ 1  20 

1530  B»SIH<Y-*S 
1540  SUBEND 
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Appendix  B 

Steepest  Descent  for  General  Analytic  Function 

Here  we  do  not  force  the  integrand  to  be  of  the  form  in  (1),  but  consider  the 
general  integral  /c  dz  f(z).  Let  f(z)  be  analytic  in  a  region  in  the  complex  plane.  The 
magnitude-squared  value  is 


M  =  |f(z) 


=  r  ♦  f* 

r  i 


(B-l) 


The  direction  of  steepest  descent  for  M  is  opposite  to  the  gradient  of  M,  which  is 

„„  3M  -  3M  -> 

7M  =  —  a  ♦  —  a  .  (B-2) 

3x  x  3y  y 

But,  from  (B-l), 


3f 

•  _J1  + 

f . 

3f .  > 

l 

r  3x 

l 

3x  , 

3f 

■  — L  + 

f . 

3f . 

l 

r  3y 

l 

3y  , 

3M 
3y  = 

Now  if  function  f  is  analytic  at  z,  then 

3f  3f .  3f .  3f 

f'U)  -  aF1*  ^  s  ^  ^ 


'  3x 


3y 


3y 


(B-3) 


(B-4) 


So  we  can  express 


4^  =  2  Re{f*(z)  f'(z) } 
>x 

^  =  -2  Im{f * ( z )  f ' (z)  } 


(B-5) 


Therefore,  the  steepest  descent  direction  for  M  has  components  that  are  the 
negatives  of  (B-5)  or,  equivalently,  are  proportional  to 


-Rc 


f  (:) 

Hz) 


T  1  f  ^ 

,  In>ifxrr 


(B-6) 


The  basic  calculation  for  determination  of  steepest  descent  directions  is  thus  seen  to 

be  H 

f'(z)/f(z)  =  ^lnf(z). 

Whenf'(zs)  =  0,  we  have  a  saddle  point  of  f  at  z  =  zs.  Near  the  saddle  point, 


ro  ?  f(=s)  .  V’os)(-  -  =s): 


1  2 

f  +  -s-f.A  for  small  A  .  (B-7) 
o  -  - 
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Then 


~  | f Q | ^  +  Rc  |fQ  ^2^}  *0T  sma11  A  ■  (B-8) 

Now  let 


f  L  =  ae 
o  2 


ioi 


A 


ie 


re 


(B-9) 


Then  (B-8)  yields 

M  =  } £q | 2  *  ar2.cos(u  +  20)  ,  (B-10) 

which  has  two  peak-  and  two  valleys  versus  0  in  a  2n  interval  (for  a  *  0);  this  is 
characteristic  of  a  saddle  point.  The  directions  of  steepest  descent,  0d,  at  z  =  zs 
occur  when 


c  ♦  20j  =  tt  or  3it  ,  9^  =  ff-  or  71  *  +  tt  .  (B- 1 1 ) 


Notice,  from  (B-9),  that 

'  =  ars{fof:|  =  ars{f  (=s}  arg{f"(“s)/f(:s)}  *  (B'I2) 

Now  let  us  investigate  the  behavior  of  the  complex  function  f(z)  near  zv,  along  the 
steepest  descent  contours.  For  0d  as  given  by  (B- 1 1),  A  in  (B-9)  becomes 


A  -  ±r  exp 


(i^). 


2  2  -ia 

A  =  -re 


and  there  follows,  from  (B-7)  and  (B-9),  for  small  r, 


rr  \  ~  c  1  2-  -id 

f(z)  =  fQ  -  2~r  £2^ 


on  steepest  descent  contour  near 
That  is,  since  the  term  in  parentheses  is  real  and  positive, 

arg  f(z)  =  arg  f ( 2 s )  on  steepest  descent  contour  near  z$. 

More  generally,  it  can  be  shown  that 


(B-13) 


(B-14) 


(B-15) 


arg  f(z)  =  arg  f(zs) 


everywhere  on  steepest  descent  contour  through  zs*  (B-16) 

Now  let  us  apply  these  general  results  to  the  special  case  w  here  f(z)  =  cxp(w(z)). 
Then  f’(z)  =  w  ’(z)  exp(w(z)),  and  f  (z)  =  0  when  w  (z)  =  0.  Thus  saddle  point 
locations  are  as  usually  stated.  Also,  as  needed  in  (B-6), 
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f'U) 

?nr 


w  (z) 


(B-17) 


and  then  (B-6)  agrees  with  (9).  Furthermore,  since 

f"(z)  =  [w"(z)  +  w'2(z)]  exp(w(z) ) 


f"(zs)  =  k"(zs)  exp(w(zs)) 

then  (B-12)  yields 


(B-18) 


a  -  arg  v"(zs)  .  (B-19) 

When  (B-19)  is  used  in  (B-l  1),  the  steepest  descent  directions  corroborate  reference 
1 ,  (7.1.8)  and  (7. 1 .19).  Finally,  (B-16)  yields 

arg  f(z)  ■  arg{exp(w(z))}  =  arg{exp(u  +  iv)}  -  v  ,  (B-20) 

meaning  that  v  is  constant  on  steepest  descent  contours;  this  agrees  with  reference  1 , 
Lemma  7.1. 

What  this  all  demonstrates  is  that,  for  a  general  given  integrand  f(z),  we  can  let 
w(z)  =  In  f(z)  and  apply  our  usual  techniques  on  w(z).  This  procedure  was  adopted 
in  the  Gaussian  exponent  example  in  the  main  text  of  this  report. 
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ABSTRACT 

A  procedure  for  determining  the  probability  density  function  or 
cumulative  distribution  function  of  measured  data  is  considered, 
whereby  the  sample  distribution  is  plotted  on  several  types  of 
transformed  axes  and  compared  with  candidate  distributions.  The 
abscissa  transformations  considered  are  linear  or  logarithmic,  while 
the  ordinate  transformations  considered  are  Gaussian  or  logarithmic. 
The  four  different  distributions  which  plot  as  straight  lines  on  the 
appropriate  combinations  of  transformations  are  displayed,  as  well 
as  being  cross-plotted  on  mismatched  paper.  Applications  to 
random  data  are  given. 
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INTRODUCTION 

In  order  to  characterize  the  probability  density  function  or  cumulative 
distribution  function  (COF)  of  measured  data,  the  sample  CDF  is  often 
displayed  on  a  cat, didate  type  of  graph  paper,  as  for  example,  a  linear 
abscissa  with  a  Gai.ssianly-transformed  ordinate.  If  the  underlying  process  is 
truly  Gaussian,  the  resulting  COF  plot  will  resemble  a  straight  line,  if  the 
number  of  independent  samples  is  large.  However,  other  CDFs  will  plot  as 
curves  on  this  type  of  paper. 

Two  frequently-used  abscissa  transformations  are  linear  and  logarithmic 
(dB  for  example).  And  two  popular  ordinate  transformations  are  Gaussian  and 
logarithmic.  Since  there  are  four  possible  combinations  of  these  two  pairs  of 
transformations,  it  would  be  worthwhile  to  have  several  common  analytic  CDFs 
plotted  on  all  four  types  of  paper,  to  serve  as  comparisons  for  future 
measured  data.  As  comparisons,  we  select  the  four  CDFs,  each  of  which  leads 
to  a  straight  line  plot  on  one  of  the  four  combinations.  In  addition,  we 
consider  a  more  general  CDF  with  several  parameters,  the  QM  distribution, 
which  occurs  for  narrowband  processing,  and  illustrate  a  transition  between 
two  of  the  basic  CDFs. 

In  order  to  guarantee  that  the  CDF  graphs  lie  in  a  standard  region  of 
plotting  space,  the  abscissas  are  normalized,  by  subtracting  the  mean  of  the 
random  variable  under  consideration,  and  dividing  by  the  standard  deviation. 
This  normalization  realizes,  in  some  cases,  a  unique  plot  for  the  CDF, 
regardless  of  the  values  of  the  parameters  of  the  CDF.  In  other  cases,  it 
reduces  the  dependence  on  the  number  of  parameters.  This  convenient  behavior 
minimizes  the  number  of  plots  and  comparisons  that  have  to  be  considered. 

Also,  for  measured  data,  the  two  required  statistics,  namely  the  sample  mean 
and  sample  variance,  can  be  simply  and  easily  computed. 
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LINEAR  ABSCISSA 


Let  random  variable  (RV)  r  have  cumulative  distribution  function  (COF) 

1?(u)=  ?rol,(r<u'))  (1) 

probability  density  function  (PDF) 

|Pr  (u)  *  (u)> 

2 

mean  mr,and  variance  op. 

For  plotting  purposes,  define  normalized  abscissa 


U-  mr 


as  a  linear  transformation.  Thus  x  measures  the  number  of  standard  deviations 
that  -Lv,e  "RV  is  frew  its  mean.  For  the  ordinate  of  the  plot,  define 

ij  =  ■jl'W}.  (4) 

where  transformation  gf  ]  is  as-yet  unspecified.  Substitution  of  (3)  in  (4) 
yields  the  characteristic  of  interest: 

Ij  =  J ^a)}  .  (5) 

Thus,  for  a  given  COF  Pr>  y  can  be  plotted  vs  x,  as  a  function  of  any 
parameters  of  ?r  that  still  remain  in  (5),  after  mr  and  or  have  been 
evaluated  and  used  in  (5). 


6 
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For  the  examples  below,  the  following  definitions  will  be  useful  (see  Ref. 


1,  (26.2),  (26.2.22),  and  (26.2.23)): 


*W  '  7^  tXf  ('  "a")  > 

f  w  -  £  a*  w . 

4.  10. 


Example  1.  Gaussian 


The  PDF  and  CDF  are 


f>,  («) *  'i~'KJir~)  ^  a|1  u> 

Tr(«)  *  ?(■¥]  ^  •«  "• 


The  mean  and  standard  deviation  of  RV  r  are 


U-« 


Mr  «  Q,  *  b  >  *HlUff  x  s  j, 


(6) 


(7) 


(8) 


Substitution  of  (7)  and  (8)  in  (5)  yields  characteristic 


ij  =  [£(*)}  "for  all  X. 


(9) 


This  result  is  independent  of  parameters  a  and  b  of  the  Gaussian  PDF, 
regardless  of  what  ordinate  transformation  g{  ]  is  employed.  This  convenient 
behavior  is  a  result  of  the  selection  of  linear  transformation  (3).  Since  no 
parameters  remain  in  (9),  a  unique  plot  will  result  for  (9),  regardless  of  the 
g[  ]  selected. 
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Example  2.  Exponential 

"R.(«)  ~  I  ~  <?Xj>(-k(u-^  -far  UZ  a. 


There  follows 


wr*£K'hi’  }  ^  =  ;  X  -  b(u-  a)  ~  I,  (n) 

Substitution  of  (10)  and  (11)  in  (5)  yields 

jj  »  J  f  I  -  e.  ]  -for  X  >  - 1.  U2) 

This  result  is  also  independent  of  parameters  a  and  b. 

Example  3.  Log-Normal 

Here,  the  natural  logarithm  of  RV  r  is  Gaussianly  distributed.  That  is, 
the  PDF  and  CDF  of  r  are 
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There  follows 

l*V 


a  ft 
e  e  , 


X 


(14) 


Substitution  of  (13)  and  (14)  in  (5)  yields 


This  result  Is  Independent  of  a,  but  it  does  depend  on  parameter  b  of  (13); 
thus  (15)  will  yield  a  family  of  characteristics,  regardless  of  the  choice  of 
ordinate  transformation  g( 


Example  4.  Power-Law 

j>, («)  =  bab/u.t+'  -fir  U2  0, 

?r(„)  =  •  "  («/«/  ua  a. 

Then  for  paramter  b  >  2, 


ab 

rt—  -  M 

nr 

b-, 

; 

*  '  b-l < 

||  b-7 

u  - 

ab 

b-l 

X  = 

b  - 1 

VtV 

Substitution  of  (16)  and  (17)  in  (5)  yields 

This  result  depends  on  b,  but  not  on  a. 
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LOGARITHMIC  ABSCISSA 

The  other  abscissa  transformation  we  are  interested  In  is  a  logarithmic 
one,  as  employed  for  example  in  dB  plots.  We  now  develop  the  basic  relations 
and  normalization  for  this  case.  When  RV  r  above  is  confined  to  positive 
values,  let  transformed  RV 

t  =  (19) 

have  CDF  and  PDF 

X  (v)  -  P*b  (t  <  v)  ,  A  W  =  Tt'  (»),  (20) 

respectively.  And  let  the  mean  and  standard  deviation  of  RV  t  be  denoted  by 
mt  and  o^. 

We  then  have,  by  use  of  (20),  (19),  and  (1),  the  CDF  of  t  as 

7>(v)  -  ?ro\>[+<  v)-  ?ro\>(A  r<v)  =  Prd(r  <  e)  -  (e.v).  (21) 

The  PDF  follows  as 

|Pt(v)  =  T^'M  =  e  jpr(e').  (22) 

Several  alternative  expressions  are  available  for  the  n-mcment  of  RV  t: 

*t  *  (^r)  “  I  V  |0t  (y)  =  du  (1  u)  j>r(«)  =  [  dv  V  e  jjr  (it)  ,  (22) 
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In  particular,  we  have 

_  X 

=  t3  -  i  .  (24) 

The  normalized  abscissa  we  adopt  for  plotting  purposes  Is  a  linear 
transformation  on  v: 


V- 

X  =  r  •  (25) 

t 

Here,  x  measures  the  number  of  standard  deviations  that  -tiie  KV  is  from  its 
mean.  The  ordinate  is  obtained  according  to 

J  =  J K+  x)j  (  (26) 

where  transformation  g£  }  is  as-yet  unspecified.  We  can  now  plot  y  vs  x,  once 
a  CDF,  Pt(  )f  is  specified. 

Example  1.  Gaussian 

The  CDF  for  the  RV  r  is  given  in  (7).  Since  RV  r  can  go  negative,  we 
cannot  apply  transformation  (19),  and  this  example  is  not  applicable. 
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Example  2.  Exponential 


We  take  the  special  case  of  a  ■  0  here;  see  (10).  (The  case  of  a  >  0  can 
be  worked,  but  Involves  exponential  Integrals.)  We  have 


Tr  (u)  =  |  -  ku)  -fcr  U  5  o, 
(u)  -  t  ku)  -ft >r  U  >  0 . 


By  use  of  (21),  we  obtain  the  CLf  of  t  as 


And  from  (23), 


T^(v)  =  I-  «  )  for  all  V. 


In  particular, 


{  =  (is-JLV)e  -  -  Y  “  ^ 

*  J  (As->4i)  e  =  +  ^  b  +  ;  (3°) 

by  use  of  Ref.  2,  4.331  1  and  4.335  1,  respectively;  V  is  Euler's  constant. 


There  follows 


~  ,  °^  =  Vi/T , 


tt/v/T 
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Substitution  of  (28)  and  (31)  in  (26)  yields 


This  result  is  Independent  of  b;  a  was  set  to  0  here. 


Example  3.  Log-Normal 


From  (13)  and  (21),  the  two  relevant  CDFs  are 

■p  (v)  =  5  i^pr)  ^ °r  *1'  y  (33) 

There  follows  immediately 

i  v-a 

a,  «;«  »,  x  -  -p  •  04) 


Substitution  of  (33)  and  (34)  in  (26)  yields 


H 


Jrw]  ftr  «l)  X. 


(35) 


This  result  is  independent  of  a  and  b;  contrast  this  with  the  linear  abscissa 
result  in  (15).  Thus  the  CDF  for  a  log-normal  RV  will  plot  as  a  unique  curve 
on  a  logarithmic  abscissa,  regardless  of  the  ordinate  transformation  gf  }  . 
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Example  4.  Power-Law 


From  (16)  and  (21), 


1?  (u)  -  I  -  (Vu)  ^>r  u  - 

T^(y)  =•  I  -  -fir  vzJk<x 


(36) 


Then 


A  « 


e.  -for  v  z 


(37) 


and  there  follows  for  the  n-th  moment  of  t. 


(38) 


In  particular, 

■fc  r  +  "  -pr  +"  (J^  j 

and  therefore 

\=TT+'*"°>  «t  *  T  ’ 

X  =  tv  -  IJla  -  I . 

Substitution  of  (36)  and  (40)  in  (26)  yields 

This  result  is  independent  of  a  and  b,  in  contrast  with  the  linear  abscissa 
result  in  (18) . 


-  -x 


-  e 


-fi 


or  X2- 


(41) 
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ORDINATE  TRANSFORMATIONS 


Thus  far,  ordinate  transformation  g{}  In  (5)  or  (26)  has  been  arbitrary. 
The  particular  two  transformations  we  are  Interested  in  are 

Gaussian  ordinate,  (42) 


I c3drithmic  ordinate.  (43) 

The  transformation  In  (42)  is  the  Inverse  CDF  for  a  normalized  Gaussian  RV,  as 
introduced  in  (6). 

We  now  consider  the  possible  combinations  of  (42)  and  (43)  with  the 
results  above  for  the  linear  abscissa  and  the  logarithmic  abscissa.  The  first 
case  is 

1.  Linear  Abscissa  and  Gaussian  Ordinate 


jW  =  f  (?) 


We  employ  (42)  on  (9),  (12),  (15)  and  (18),  to  get 
|- 1  Ij  =  X  "for  all  X 


1-2  |j  *  £(l-  x)  -for  x  >- 


1-3 


1-4- 


(44) 


V 

r  1  ('- (mI^F +Tr)b)  x>- 


lb'  bx 

e  -e 


k-2 
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2.  Linear  Abscissa  and  Logarithmic  Ordinate 


In  this  case,  we  use  ordinate  tranformation  (43)  on  (9),  (12),  (15)  and 
(18),  to  obtain 


2-1  ^  JW)  -(troll  X 

ij  =  H-  X  -for  x  2  -  I 


2-2 


2- 3 


y  -A  (l - 1 (*  V^-ek"  +  e^))) 


-for 


x  >  - 


(45) 


1  J>  +^i)  ^  x>-V'fer_ 


3.  Logarithmic  Abscissa  and  Gaussian  Ordinate 

Here  we  apply  (42)  to  (32),  (35),  and  (41);  the  Gaussian  example  is  not 


appl icable. 

3-i 

Not  <jpplicol?le 

■  -  TT  v  i  v  v 

3-2 

r*( 

I  -  e.^"  for  oil 

3-3 

r * 

•for  all  X 

J-4 

r*l 

1  -  e  1  *)  “for  y  >  - 1 
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4.  Logarithmic  Abscissa  and  Logarithmic  Ordinate 

In  this  case,  we  employ  (43)  on  (32),  (35),  and  (41). 


4-1 

Njot  cyclic*  tie 

4-* 

H  * 

"for  $11  X 

(47) 

4-9 

Jj  -  -Jk  0  ■  £(*)) 

"for  ®ll  X 

4-4 

ij  =  1  4  \  for  x  > 

-  1 
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GRAPHICAL  RESULTS  FOR  THE  FOUR  PDF  EXAMPLES 

In  figure  1  are  collected  the  results  for  a  linear  abscissa  and  Gaussian 
ordinate.  In  particular,  figure  1A  contains  the  plots  for  RVs  with  a  Gaussian 
PDF  (case  1-1  of  (44))  and  an  exponential  PDF  (case  1-2  of  (44)).  The 
abscissa  limits  correspond  to  +3  standard  deviations  from  the  mean,  while  the 
ordinate  limits  are  for  probability  values  .001  and  .999.  This  is  the  type  of 
paper  on  which  a  Gaussian  COF  plots  as  the  unique  straight  line  indicated. 

Figure  IB  gives  the  situation  for  a  log-normal  PDF  (case  1-3  of  (44)),  for 
various  values  of  parameter  b.  The  curve  labelled  b  -  0+  is  the  limit  of  1-3 
in  (44)  as  b— *0+,  namely 

y  X  -for  q|)  V  <«k-*0+.  (48) 

Thus  as  the  spread  of  the  log-normal  RV  (see  (14))  tends  to  zero,  the  plot 
tends  to  the  straight  line  for  a  Gaussian  RV;  see  figure  1A. 


Figure  1C  contains  the  power-law  PDF  (case  1-4  of  (44))  for  various  values 
of  b.  The  curve  labelled  b  =  +«o  is  the  limit  of  1-4  in  (44)  as  b-»  +  oa  , 
namely 


H 


f  (l-  O 


x>-l 


ad  \>  -* 


(49) 


This  is  the  exponential  PDF  plotted  in  figure  1A. 
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The  results  for  a  linear  abscissa  and  a  logarithmic  ordinate  are  given  in 
parts  A,  8,  and  C  of  figure  2.  Now  the  exponential  PDF  plots  as  a  straight 
line;  see  figure  2A.  The  log-normal  PDF  is  displayed  in  figure  2B.  The  curve 
labelled  b  ■  0+  is  the  limit  of  2-3  in  (45): 

U  — •  - Jn  (l  -  ?(*))  4v  «ll  x  «S  l  -  Ot  (50) 

This  Is  also  the  Gaussian  plot  In  figure  2A.  The  power-law  PDF  is  given  in 
figure  2C.  The  curve  labelled  b  ■  +  is  the  limit  of  2-4  in  (45): 

j  — *  |  4-X  -for  X  > -I  $5  (51) 

This  is  also  the  exponential  plot  in  figure  2A. 

For  a  logarithmic  abscissa  and  Gaussian  ordinate,  the  equations  in  (46) 
are  plotted  on  figure  3.  Here,  the  log-normal  PDF  yields  a  straight  line. 

There  are  no  parameters  that  have  to  be  investigated  on  this  type  of  plotting 
paper,  if  the  PDF  is  exponential,  log-normal,  or  power-law;  see  (46).  Other 
PDFs  could,  of  course,  still  involve  parameters  that  are  not  suppressed. 

The  analogous  results  for  a  logarithmic  abscissa  and  logarithmic  ordinate 
are  displayed  in  figure  4.  This  time,  the  power-law  PDF  yields  the  straight 
line.  Again,  no  parameters  of  these  particular  PDFs  are  involved,  as  they 
have  disappeared  in  (47). 


22 


TM  No.  811184 


F'0Ufe  2&.  Loj-  Normal  Pt>F 

FigM^e  2.  Plots  -£r  Linear  /HjO  ssa  0x4  L-Oj  ar.Hmic  0r<ji**4e 
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Figure  3.  Pl«b  &r  Log  an -Hi  miC  Abscissa,  cmd  (xau55i'an  C^cji^ote 
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The  last  analytical  example  we  consider  Is  the  QM  distribution  with  a 
CDF  given  by  (Ref.  3) 

TJ(u)  *  I”  d,/2au. )  -fir  K?0,  (52) 


and  PDF 


H  >  0. 


(53) 


For  M  ■  1,  (53)  Is  closely  related  to  the  Rice  PDF.  On  the  other  hand,  for 
d  «  0,  It  reduces  to 


a 


(gu)  axp(-qu) 

(m-i)! 


■fcr  ^  0, 


(54) 


which  is  the  chi-squared  PDF  with  2M  degrees  of  freedom.  Thus,  (53)  is  a 
general  form  which  subsumes  several  common  cases. 


The  characteristic  function  of  RV  r  is  (Ref.  2,  6.631  4) 


(?)  *  ‘  (|-i?/a)Me*j>(4 


Therefore 

Ja  i  (?)  = 


Mi  (1-4)  +-  4- 


,'7/fl 
I- (T/a 


(55) 


(56) 
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The  cumulants  of  RV  r  are  Immediately  available: 

f°r  11 


In  particular, 


mr  55  VW  r  “ir(h+  4r)  ; 

vw -  d*)- 


(57) 


(58) 


Then  from  (3)  (on  a  linear  abscissa), 

ciu-M  -  *4“ 

X  =  - - r  - 

yM4  d1 


(59) 


while  from  (5)  and  (52), 


=  *T)j  “for 


2Mf  d* 

T(m7F 


(60) 


This  result  is  independent  of  scaling  parameter  a  in  (52)  and  (53);  however, 
it  depends  on  both  M  and  d  as  indicated. 


Equation  (60)  is  plotted  on  a  linear  abscissa  and  Gaussian  ordinate  for 
M  *  1,  2,  4,  10  in  the  four  parts  of  figure  5,  as  parameter  d  takes  on  the 
values  0,  1,  2,  4,  8.  The  curve  labelled  d  =  +  <*  is  the  same  as  that  for  a 
Gaussian  PDF.  As  d  varies  fr»m  0  to  +  »  ,  the  CDF  sweeps  out  the  region 
between  the  chi-squared  PDF  (given  in  (54))  and  the  Gaussian  PDF.  Also,  for  a 
given  value  of  d,  larger  M  values  result  in  a  PDF  curve  that  is  more  nearly 
Gaussian. 
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RAYLEIGH  PDF 


The  PDF  and  CDF  for  a  Rayleigh  RV  r  are 


frW)  *  -jf  «r(-  iy)  •fi>r  m  2  °> 

"?,(«)  =  I-  ■f“r  ua0' 


The  n-th  moment  of  r  Is 


t"  7  r(H, 


(61) 


(62) 


leading  to  (for  a  linear  abscissa) 


\ 


x  > 


Then  (61)  and  (5)  yield 


J  =  3^1"  ^f("5"(l+^']  x))]  X2- 


4-' 


(63) 


(64) 


This  result  is  independent  of  scaling  parameter  b  in  (61). 


If  g[ }  is  taken  as  the  logarithmic  transformation  given  by  (43),  then  (64) 
takes  the  special  form 

U  =j(\+‘J¥-'  x)  -fer  K3-\jJ^r  ■  (65) 

That  is,  the  Rayleigh  PDF  plots  as  a  parabola  on  a  linear  abscissa  and 
logarithmic  ordinate. 
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For  a  logarithmic  abscissa,  (19),  (21),  and  (61)  yield  the  CDF  for  RV  t  as 


"£(.*)'  I  -  *xp(- 


(66) 


The  n-th  moment  of  RV  t  Is,  according  to  (23)  and  (61), 

¥  =  C du  *T 

=  J^is  e"5  |J<  +  i’Jis]  . 


There  follows  (in  a  fashion  similar  to  (30)) 


Then  (26)  and  (66)  yield 


ir- 

2^ 


(68) 


(69) 


which  is  independent  of  b. 


But  this  result  is  identical  to  (32)  for  an  exponential  PDF  on  a 
logarithmic  abscissa.  The  reason  for  this  is  that  the  Rayleigh  RV  r  here  is 
related  to  the  exponential  RV  r  in  (27)  according  to  (using  parameter  E  for 
(27)) 


r 


(70) 
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Thus 

-t-Jir  +  ,71, 

That  is,  logarithmically-transformed  variates  t  and  are  linearly  related  to 
each  other,  and  the  normalization  in  (25)  will  treat  these  two  quantities 
equally. 

More  generally,  for  a  RV  r  with  an  arbitrary  PDF,  consider  the 
non-li nearly  related  RV 


t> 

r  =  *  r  .  (72) 

Then  the  logarithmically-transformed  RVs  are  related  according  to 

-  Jih'  +  vJi  r  -  +•  >*  i  •  (73) 

But  transformation  (25)  will  result  in  exactly  the  same  plot  for  CDF  Pt(  ) 
as  for  P|(  ).  This  holds  for  a  logarithmic  abscissa,  regardless  of  what 
ordinate  transformation  g{ J  is  employed.  (The  analoguous  situation  for  a 
linear  abscissa  is  for  the  RVs  to  be  related  according  to  r  M  +pr). 


36 


TM  No.  811184 


SIMULATION  RESULTS 


N  Independent  Gaussian  RVs  {r^  with  an  arbitrary  mean  and 
standard  deviation  were  generated  (Ref.  4).  The  sample  mean  ft  and  sample 
standard  deviation  $■  of  the  set  was  computed,  and  each  RV  was  modified 
according  to 


A 


-ftr  1 5  k  *  N, 


(74) 


giving  a  new  set  with  zero  sample  mean  and  unit  sample  standard  deviation. 

Then  the  new  set  [r^  was  ordered  according  to  size,  from  most-negative 
to  most-positive,  giving  set  ^r^J.  The  sample  CDF  was  then  realized 
by  setting  it  to  value  (k  at  abscissa  rk,  for  1  £  k  <  N,  and  drawing 

straight  lines  between  these  points.  The  results  for  a  linear  abscissa  and 
Gaussian  ordinate  are  given  as  the  jagged  curves  in  figure  6,  with  N  »  100  in 
figure  6A,  and  N  ■  1000  in  figure  68.  The  straight  line  overlaid  on  the  plots 
is  the  Gaussian  PDF  result. 


When  scaled  and  shifted  exponential  RVs  are  generated  and  subjected  to  the 

same  procedure,  the  results  in  figure  7  are  obtained.  The  smooth  curve 

overlaid  on  the  jagged  sample  CDF  plots  is  the  exponential  PDF  as  given  in 

figure  1A.  The  results  in  figures  6  and  1  allow  for  ready  confirmation  of  the 

character  of  the  sample  CDF.  A  sample  program  for  the  procedure  is  given  in 
the  appendix. 
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DISCUSSION 


Additional  candidate  PDFs  or  CDFs  can  be  added  to  those  given  herein, 
thereby  building  up  a  catalog  of  comparison  cases.  Also,  other  ordinate 
transformations,  like  the  arc  tanh  function  utilized  in  coherence 

transformation  (Ref.  5),  can  be  Included.  In  this  way,  an  experimental  sample 
CDF  can  be  readily  compared  with  several  ideal  analytical  forms,  in  an  attempt 
to  easily  find  a  reasonable  characterization  for  the  statistics. 
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APPENDIX 

PROGRAM  FOR  SIMULATION  RESULTS 


10  kANDOMIZE  SQR  < .  6 ) 

20  N* 1 0O 

30  dih  va:  loo©), bu:  1O0O) 

40  REDIM  Y<l:N),B<l:N) 

30  FOP  1*1  TO  N 

60  Y(I)*-L0G<RND>*3.1+.57 

70  NEXT  I 

30  FOR  1*1  TO  N 

30  Sl-Sl+YCI) 

10G  S2*S2+Y<.  I  >A2 

110  NEXT  I 

120  Me*n«Sl^N 

130  V*r»<S2-Sl*Sl/N)/ (N-l ) 

140  Stdd*v*SQR< Var ) 


150 

FOR  1*1  TO  N 

160 

Y < I ) *< Y< I )-M**n)/ Stddeu 

170 

NEXT  I 

ISO 

FOR  K»1  TO  N 

ISO 

A* 1 E30 

2O0 

J*0 

210 

FOP  1*1  TO  N 

220 

IF  Y < I ) ) A  THEN  250 

230 

A*Y (.  I ) 

4  0 

J*I 

250 

NEXT  I 

260 

B  OO  *  A 

270 

Y  <  J  )  *  3  E  3  3 

230 

NEXT  K 

230 

PLOTTER  IS  " 

GRAPHICS" 

30O 

GRAPHICS 

310 

XI*- 3 

320 

X2*3 

3  30 

Yl*FNIrvphi  < 

.  0O1 

340 

Y  2  *  F  N I  rv  ■  p  h  \  ' 

.  333 

350 

SCALE  X1.X2, 

VI ,  Y2 

360 

FOR  X*X1  TO 

X2 

370 

MOVE  X , Y 1 

330 

DRAW  X , Y 2 

330 

NEXT  X 

4O0 

PENUP 

!  RANDOM  NUMBER  SEED 
!  NUMBER  OF  SAMPLES 


!  EXPONENTIAL  PDF, 

!  SCALED  AND  S H I F T E D 


'  NORMALIZE 
1  SORT  INTO  E  APR  A  ( 


•  GAUSSIAN  0  °  D  I  ( 4  7  T. 
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41©  DATA  .081, .802,  .005, .01, .0i, .05, . 1, .2,  .  3.  .4,  .5 
420  DATA  . 6, . 7, .8,  .  9, . *5, . 9S, .99, . 995, .995,  .  99? 

430  DIM  0<i:21> 

440  READ  0< * ) 

450  FOP  1*1  TO  21 
4€0  T*FNIm>phi  (0<  I  ;■  ) 

470  MOVE  XI, T 

480  DRAW  X2,T 

490  NEXT  I 

500  PENUP 

510  FOR  X*- . 99  TO  X2+.03  STEP  .03 

520  Y*FNInuphi ( 1-EXP< -1-X> >  1  EXPONENTIAL  3 Dr 

530  PLOT  X, Y 

540  NEXT  X 

550  PENUP 
580  FOR  1=1  TO  N 

570  Y=FN I nvphi <<I-.5>^NJ  !  SAMPLE  CDF  ORDINATE 

580  PLOT  B < I ) , Y 

590  NEXT  I 

800  PENUP 
810  END 

820  ' 

830  DEF  FHIrv,phi  OO  '  INVERSE  PHI  r UNO  f .  ON 

640  IF  •:X>*0>  AND  CX<*1>  THEN  670 

650  PRINT  "ARGUMENT  *  ;  1 1 ; “ I S  DISALLOWED" 

660  ST  0 P 

670  IF  'X>0>  AND  <X<1>  THEN  700 
680  P*9. 99999999999E99*1 2*X-1  ' 

690  GOTO  760 
7O0  P*X 

710  IF  .0.5  THEN  P*.5-CX-.5> 

720  P*SQR< -2*L0G<P > > 

730  T=  1  +  P  *  •  1 . 4327S8  +  P-*  .  13926  9  *?'■*.  OO 1  30S  1 

740  P=P- • 2.51 55 17+P*< . 802853+P* . O 1 O 328 > •  T 

750  IF  X< . 5  THEN  P=-P 
760  RETURN  P 


770  FNEND 
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On  the  Distribution 
Of  a  Chi-Squared 
Variate  Raised 
To  a  Power 


A.  H.  Nuttall 

ABSTRACT 

The  probability  distribution  of  a  chi-squared  variate  tends  toward 
normality  as  the  number  of  degrees  of  freedom  increases.  However, 
some  powers  of  the  chi-squared  variate  tend  to  normality  much 
faster.  For  example,  the  1/3  power  of  a  chi-squared  variate  of  just  4 
degrees  of  freedom  is  virtually  normal  over  the  whole  range  (.001, 
.999)  of  probabilities.  Inspection  of  the  cumulants  reveals  that  the  1/3 
power  is  best  for  minimizing  the  third  and  fifth  cumulants,  and  nearly 
optimum  for  the  fourth  cumulant. 
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INTRODUCTION 

The  approach  of  a  chi-squared  variate,  z,  to  normality  as  the  number  of 
degrees  of  freedom,  K,  increases  is  well  known.  However,  it  has  been  observed 
that  the  square-root  of  z  tends  to  normality  faster;  see  [1,  p.  79,  par.  4], 
[2,  pp.  251  and  420],  and  [3,  pp.  371-2].  Furthermore,  the  cube  root  of  z 
tends  even  faster  to  normality,  based  on  consideration  of  the  low  order 
cumulants;  see  [3,  pp.  371-4]  and  [4].  Here  we  will  look  at  the  cumulative 
distribution  function  and  cumulants  of  zv  for  various  values  of  power  v  and 
degrees  of  freedom  K,  and  furnish  a  quantitative  measure  of  the  discrepancy 
from  the  Gaussian  distribution. 


1-4 
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CHI-SQUARED  VARIATE  OF  K  DEGREES  OF  FREEDOM 

Let  for  1  <.  k  ±  K,  be  K  independent  Gaussian  random  variables  with 
zero  mean  and  unit  variance.  Then 


z  t  X2*  x2  +  x2  +  ...  +  x2  (1) 

is  called  a  chi-squared  variate  of  K  degrees  of  freedom.  The  characteristic 
function  of  z  is 


f,if)  -  exp(ifz)  -  (l-i2y)"K/2  ; 


(2) 


the  probability  density  function  of  z  is* 


K  _  1 

•  P>)-4t2 - -P— ^  for  u  >  0  ;  (3) 

2  2K/^P(K/2) 

and  the  cumulative  distribution  function  of  z  is  [7,  eq.  6.5.5  a.wJ  2C.4.Q 


u 


-  1 


Pz ( u )  a  Prob  {z  <  u}  »  dt  - exp^~t^2^  s  P(uJk)  for  u  >  0.  (4) 


P(K/2) 


The  cumulative  distribution  function  is  also  expressable  in  terms  of  the 
incomplete  gamma  function  and  confluent  hypergeometric  function  (for 
non-integer  K)  as  [7,  eq.  6.5.2] 


P»  =  Y 


&  i)M) 


(?)  exp(_  ?)ifi  (1;  1  * t)/i r(|  * i)  •  <s> 


*  Tne  probability  density  functions  and  cumulative  distribution  functions 
encountered  here  are  zero  for  negative  arguments,  except  for  §  in  (8). 
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The  general  v-th  moment  of  chi-squared  variate  z  is 


zv  »  2V 


rfH 

~fW 


for  v  >  - 


(6) 


and  the  cumulants  are 


xz(n)  »  K  2n-1(n-l) I 


for  n  »  1,  2,  . . . . 


(7) 


Special  cases  of  (4)  are 


P(u|l)  -  2$(YIT)-1,  $U) 

P(u|2)  a  l-exp(-u/2). 


-x2/ 2), 


(8) 


and  more  generally,  we  have  recursion 


i-  1 

P(u|K)  •  P(u| K-2)  -  (f)  exp(-u/2)/r(K/2)  for  K  >  3.  (9) 

The  function  $  in  (8)  is  the  cumulative  distribution  function  for  a  zero-mean 
unit-variance  Gaussian  random  variable. 
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v-TH  POWER  OF  A  CHI-SQUARED  VARIATE 
Let  random  variable  v  be  the  v-th  power  of  the  chi-squared  variate: 


v-z'’-^.2)''  ,  (10) 

where  v  >  0.  The  cumulative  distribution  function  of  v  is,  upon  use  of  (4), 

Py(u)  »  Prob{v  <  u]  -  Prob{zv  <  u]  ■  Prob^z  <  u^v] 

»  p2(u1/v)  -  p(u1/v|k)  ,  (11) 

in  terms  of  the  cumulative  distribution  function  of  a  chi-squared  variate. 

The  moments  of  v  are,  using  (10)  and  (6), 


In  particular,  the  mean  and  standard  deviation  of  v  are 


On  a  normalized  abscissa  x,  we  therefore  have  to  consider  the  distribution 
[5,  eq.  5] 


here  we  used  (11). 


(14) 


Plots  of  cumulative  distribution  functions 

For  v  =  1,  v  in  (10)  is  simply  the  chi-squared  variate  of  K  degrees  of 
its  cumulative  distribution  function  (14)  is  plotted  in  figure  1. 
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NUMBER  OF  STANDARD  DEVIATIONS  FROM  MEAN 


u re  |.  of  CW~  Squar'd  Var £ 
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The  curve  labelled  K»*o  is  the  Gaussian  cumulative  distribution  function  and  is 
a  straight  line  on  this  paper  with  a  normal  ordinate.  The  approach  to  this 
Gaussian  curve  progresses  rather  slowly  with  K,  especially  on  the  tails. 

Plots  of  the  cumulative  distribution  function  of  v  for  v  «  1/2,  1/3,  1/4, 
1/5,  as  given  by  (14),  are  presented  in  figures  2-5,  respectively.  The 
improvement  in  approach  to  normality  for  v  =  1/2  in  figure  2  over  that  for  \>  = 
1  in  figure  1  is  significant.  However,  that  for  \>  =  1/3  in  figure  3  is 
remarkably  good;  in  fact,  the  cumulative  distribution  functions  for  K  >  8  in 
figure  3  are  virtually  on  top  of  the  Gaussian  curve  in  the  entire  range  of 
probabilities  (.001,  .999). 

When  we  continue  with  the  trend  toward  smaller  v  values  in  figures  4  and 
5,  the  cumulative  distribution  functions  begin  to  deviate  further  from  the 
Gaussian  curve,  at  both  ends  of  the  probability  scale.  For  example,  for 
v  »  1/5  in  figure  5,  the  cumulative  distribution  functions  are  all  above  the 
Gaussian  result,  whereas  for  \>  =  1/3,  they  were  all  below.  All  the  cumulative 
distribution  functions  approach  the  Gaussian  curve  as  K  increases,  but  the 
approach  is  not  monotonic  with  K. 

Finally,  for  a  v  value  larger  than  1,  namely  v  =  5/4,  the  plot  in  figure 
6  reveals  even  greater  discrepancy  from  the  Gaussian  curve  than  for  the 
chi-squared  variate  itself,  i.e.,  v  =  1  in  figure  1. 


Cumul ants 

Another  measure  of  the  non-Gaussian  character  of  a  random  variable  are 
the  normalized  cumulants.  The  cumulants  of  v  [3,  p.  70]  are,  in  terms  of  (12), 


xv(2)  =  u2-u^  *  (v-v)' 


*v(3)  =  (v-v)3  =  u3  -3^1^  +  2u3 


*v(4)  =  ( v-v ) 4  -  3x^(2)  =  u4  -  4u3uj  -  3u2  +  -  6u^ 


Xy(5)  =  -  10u3u2  +  +  30u2u^  -  60u2u3  +  24u3 


9 


(15) 
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pi^re  3.  V  =  2  V  P  -  ^ 
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NUMBER  OF  STANDARD  DEVIATIONS  FROM  MEAN 
Flours.  5".  of  V-  2  4r  P  -  f 
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The  nonnalized  n-th  cumulant  of  v  is 

\(n) 

'  (16) 

This  quantity  is  plotted  in  figures  7,  8,  9  for  n  *  3,  4,  5  respectively.  The 
normalized  third  cumulant  in  figure  7  is  near  0  for  v  *  1/3  when  K  is  large. 
Thus  this  particular  measure  of  non-Gaussianness  is  minimized  by  choosing 
va»  1/3. 

The  normalized  fourth  cumulant  in  figure  8  is  smallest  in  the  range 
(1/6,  1/2)  for  v,  when  K  is  large.  It  appears  to  be  approaching  zero  for 
v  «  1/2  and  v  *  1/4;  however,  it  is  still  quite  small  for  v=l/3. 

The  normalized  fifth  cumulant  in  figure  9  approaches  zero  (as  K 
increases)  at  v  -  1/6,  1/3,  7/12,  and  is  nearly  zero  over  an  extended  range 
of  \>. 


Approximation  to  Cumulative  Distribution  Functions  of  v  and  z  ' 


For  v  =  1/3,  since  random  variable  v  is  nearly  normal,  we  have  as  an 
approximation  to  its  cumulative  distribution  function, 

p»(u)*  iffi)  •  (17) 

where  $  is  the  Gaussian  cumulative  distribution  function  defined  in  (8).  And 
from  (12)— (13),  we  have 


m 


v 


(19) 


1  C 
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Figure  1-  TV»r«i  C^mulav^ 
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Then  the  cumulative  distribution  function  of  chi-squared  variate  z  is,  upon 
use  of  (10)  (with  v  ■  1/3)  and  (17), 

Pz(u)  ■  Prob  (z  <  u}  -  Prob  {v^  <  uj  ■  Prob  {v  <  u*^J 


(u1/3)  .  § 


This  is  similiar  to  the  procedure  in  [6,  p.  597,  19.3-2]. 


Asymptotic  Behavior  of  Moments  of  v 


The  n-th  moment  of  v  «  zw  ■  Q(?)w  was  given  in  (12).  By  use  of 
[7,  eq.  6.1.47],  identifying  z-*K/2,  a-*nv,  b-»0,  we  have,  as  K  tends  to-*-** 
in  all  the  following  asymptotic  relations, 


u  «  K' 

Mn 


Ti  ♦  ♦  a(a-u.ia-2).(3a-1; 

L  K  6KZ 


+  0 ( K— 3 ) 


un  «  (K-l+a) a  [1+0(K’Z)]; 


here  a  is  to  be  interpreted  as  nv.  See  also  [3,  pp.  371-4]. 


Particular  cases  of  (20)  are  first  moment 
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or 

u2  -  (K-1+2v)2v  [1*0(K’2)] 
There  follows  for  the  variance  of  v  >  zv, 


,2  -  4  -  K2-1  2v2  [l*  .  0(K-2)] 


U2  -  u2  -  2v2  K',(K-1+3v)v-1  [1+0(K"2)] 
Particular  cases  of  (21)  and  (23)  are: 

“l  *  (K  -|)U2  Ci*o(k~2)3 
u2 '  "l  ■  7 

these  generalize  [1,  p.  79,  par.  4],  where  the  variable  treated  is 
V&  rather  than  "VT1.  Also 


The  approximation 


■i  *  (K  - 


!f 


for  v  =  1/3 


is  within  11  for  K  >  3,  and  the  approximation 


u2  -  u2  h  !  K"1/3  for  v  =  1/3 


(22) 


(23) 


(24) 


(25) 


(26) 


(27) 
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Is  within  17.  for  K  >  5.  The  same  errors  hold  for  v  ■  1/2  also;  i.e. 
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STATISTICS  OF  THE  SAMPLE  GEOMETRIC  MEAN  OF  A  SET 
OF  MULTIPLES  OF  CHI-SQUARED  VARIATES 


This  section  will  deal  with  the  product  of  a  number  of  independent 
chi-squared  variates,  instead  of  the  power  of  a  chi-squared  variate.  This 
operation  (of  extracting  the  sample  geometric  mean  of  a  set  of  random 
variables)  is  encountered  in  maximum  likelihood  estimation  of  unknown 
parameters  of  a  candidate  cumulative  distribution  function  of  chi-squared  form. 

The  probability  density  function  of  a  multiple  of  a  chi-squared  variate, 
w,  of  K  degrees  of  freedom,  is  from  (3), 

K  .  x 

pw(“)-iT77 - 6XP-^— /8^  for  u>°  •  (29) 

w  &Kld  T(K/2) 

The  cumulative  distribution  function  is,  upon  use  of  (4), 


p„(U) 


(30) 


Now  let  wn,  for  1  <  n  <  N,  be  a  set  of  independent  chi-squared 
variates,  each  with  the  probability  density  function  in  (29).  The  sample 
geometric  mean  of  the  set  {wn}  is 


1/N 


g  =  (w1  w2  ...  wN)  =  exp 


w1  +  ...  +  jfo  wN 


(31) 


He  are  interested  in  the  cumulative  distribution  function  of  g.  Define 


>54 


n=l 


.yn  »  (-<*><  X  <  +ae>) 


(32) 


where 


yn  5  J?n  wn  for  1  <  n  <  N 


(33) 
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The  characteristic  function  of  any  one  of  the  [yp]  is,  from  (33)  and  (29), 


■■ry 

fy(?)  -  exp(ifcy)  -  exp ( i  T  M  w)  »  wiy 


f 


if+  l  -1 


du 


exp(-u/s)  fl1f  rfl  +  i^) 


,0  eK/ii  P(K/2) 

Therefore  the  characteristic  function  of  jf  in  (32)  is 


N  _  Jf 


fjj(f)  -  expniPfT  -  [fy(*/N)]"  «  6 


r(i  * '  t) 


nN 


(34) 


(35) 


Finally,  the  cumulative  distribution  function  of  the  sample  geometric  mean  g 
is,  by  use  of  (32), 


Pg(u)  ■  Prob  [g  <  uj  »  Prob  <  ^n  u] 

*  Pg  (in  u)  for  u  >  0  (36) 

The  only  numerical  step  required  to  find  the  cumulative  distribution  function 
of  5  is  that  from  characteristic  function  £  in  (35)  to  its  cumulative 
distribution  function  ^  required  in  (36);  the  techniques  in  [8,  9]  are  useful 
in  this  regard.  For  N  «  2,  a  simpler  approach  for  the  cumulative  distribution 
function  of  g  is  available  in  [10,  p.  17]. 
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SUMMARY 

The  near-Gaussian  character  of  the  one-third  root  of  a  chi -squared 
variate,  with  4  degrees  of  freedom  or  more,  leads  to  the  simple  approximation 
for  the  cumulative  distribution  function  of  a  chi-squared  variate  as  given  by 
(19).  The  required  parameters  are  given  by  (18)  or  approximations  ( 25 ) -( 27 ) . 
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ABSTRACT 

An  approximation  for  the  threshold  signal-to-noise  ratio,  at  which  the 
corelator  performance  estimate  deviates  from  the  Cramer-Rao  lower 
bound,  Is  derived  as  a  function  of  the  time-bandwidth  product,  TB, 
and  the  probability  of  an  anomaly,  P.  The  dependence  on  P  is  weak, 
and  the  dominant  behavior  of  the  threshold  signal-to-noise  ratio  is 
according  to  (TB)'1^. 
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INTRODUCTION 

Estimation  of  the  time  delay  between  two  signals  received  at  two  sites  in 
uncorrelated  additive  noises  is  often  accomplished  by  filtering  and  cross- 
correlating  the  two  received  waveforms  and  locating  the  peak.  Extensive 
results  on  the  performance  of  this  technique  are  given  in  [1-10].  It  is  often 
observed  that  the  correlator  performance  deviates  rather  abruptly  from  the 
Cramer-Rao  lower  bound,  in  terms  of  the  variance  of  the  time  delay  estimate, 
when  the  input  slgnal-to-noise  ratio  is  decreased  below  a  threshold  value. 

Here  we  derive  an  approximation  to  this  threshold  signal-to-noise  ratio,  which 
is  valid  over  a  wide  range  of  TB  and  P,  where  T  is  the  observation  time,  B  is 
the  signal  bandwidth,  and  P  is  the  probability  of  an  anomaly  [4]. 


1-5 
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TECHNICAL  CONSIDERATIONS 

The  starting  point  is  the  variance  of  the  time  delay  estimate  as  given  in 

[«]: 


„2  -  P  r|/3  *  (1-P)  4 . 4  *  P<t|/3  -  4),  (1) 

where 


P  -  1  -  [dx  4>(x-a)  [{(BX)]^1  .  (2) 

£ 

Her?  P  is  the  probability  of  an  anomalous  estimate,  TQ  is  the  segment  length 
used  in  the  correlation  processing  technique,  and  o2R  is  the  Cram6r-Rao 
lower  bound  on  the  variance  of  the  time  delay  estimate.  The  functions  in  (2) 
are 


♦(x)  «  (2w)*1/2  exp(-x2/2)  , 

x 

f(x)  *  f  dt  f(t)  ,  (3) 

and  we  have  parameters 


M 


4T0B, 


(4) 


where 


p  S  S/N 

R  =  =  M 


(5) 


S  and  N  are  the  input  signal  and  noise  powers  at  the  correlator  inputs.  Thus 
S/N  is  the  input  signal-to-noise  ratio  on  one  element. 


Now  if  the  probability  of  an  anomaly,  P,  is  zero,  the  variance  of  the 

2 

time  delay  estimate  in  (1)  is  simply  a£R.  However,  as  P  increases 

2  2 
slightly  from  zero,  the  Tn/3  term  in  (1)  rapidly  takes  over  and  a  increases 

More  accurately,  •tha.  Search  interval  ■Bie  Cerrelrtj  on  estimate 
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significantly.  Since  this  effect  Is  controlled  by  P,  it  is  of  interest  to 
determine  at  what  input  signal-to-noise  ratio,  S/N,  the  probability  P  begins  to 
deviate  from  zero.  Mathematically,  given  a  small  P  >  0,  we  must  solve  (2)  for 
S/N,  where  the  parameters  are  listed  in  (4). 

From  [10,  figures  1-4],  It  Is  seen  that  for  large  T8,  in  order  to  realize 
a  small  value  of  P,  the  required  input  signal-to-noise  ratio  is  much  less  than 
1.  In  fact. 


for  TB  >>  1  (6) 

is  a  ballpark  figure,  which  we  want  to  improve  on.  Then 

a  ~  8,  8  ~  1  for  TB  >>  1  (7) 

Since  ^(x-a)  peaks  at  x»o,  §(ex)  in  (2)  takes  its  relevant  argument  values  at 
sx  ~  Sa  ~  8. 


S  _  8 
N 


Now  for  as  large,  we  have 


Blsx)]*-1  .  [1  -K-ex)]11-1  s  1  -  (H-l)lt-flx)  . 

Then  from  (2), 

P  ~  1  -Jdx  d(x-a)[l-(M-l)J(-8x)]  =  (U-l)Jdx  <f>(x-a)  <$(-3x) 

-  (M-l)  | dt 4<t)  $(-Bt-afl)  -  (M-l)$| 

Here  we  used  the  integral  result 

dx  exp  ^P2x2  +  qx)  $(ax+b)  =  ^  exo^-^^^P 

for  p  4  0. 


(8) 


(9) 


(10) 
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The  approximation  in  (9)  is  actually  an  upper  bound;  this  follows  from 
(8)  and  the  observation  that 


(l-u)^1  >  1-(M— 1  )u  for  0  <  u  <  1  ,  (11) 

which  is  simply  a  statement  that  the  left-hand  side  of  (11)  is  greater  than 
the  tangent  at  u  *  0.  Thus 

with  no  approximations. 


By  now  employing  the  parameter  definitions  in  (4),  (9)  becomes 

P  *  (4T0B-1)  lLfr-jsL=i \  •  (13) 

l  7l+R2/2  J 

If  we  let  I  be  the  inverse  function  to  and  define 


then  (13)  can  be  solved  for  R  according  to 


V-yfc/2 

Then  (5)  yields  the  input  si gnal-to-noise  ratio  required  as 


(15) 


S 

N 


(16) 


in  terms  of  the  quantity  defined  in  (14).  For  large  TB,  y  is  small,  and  (16) 
and  (14)  yield  approximation 


(17; 


8 


TM  No.  831065 


This  result  Is  reminiscent  of  [9,  eqs.  2  and  3],  where  we  must  note  that 
SNR  there  Is  proportional  to  the  square  of  S/N  here;  see  [9,  eq.  1].  Also 
those  results  are  Independent  of  the  probability  of  anomaly,  by  virtue  of  the 
choice  of  threshold  variance  as  being  twice  the  Cram£r-Rao  lower  bound. 

The  Inverse  function  $  In  (14)  and  (17)  Is  a  relatively  weak  function  of 
its  argument;  thus  the  dominant  behavior  of  the  required  threshold 
signal-to-nolse  ratio  is  according  to  (TB)"^,  i.e.,  -1.5  dB  per  doubling 
of  the  TB  product. 

A  short  table  of  the  required  S/N  as  calculated  from  (14)  and  (16)  is 
given  in  table  1  for  B  ■  100  Hz,  T  -  1/8  sec,  and  P  -  10"®;  similar 

C  ° 

results  for  P  -  10“3  are  listed  in  table  2.  Comparison  of  these  tabular 
results  with  [10,  figure  2]  reveals  that  probability  of  anomaly  P  »  10“6 
marks  the  breakpoint  between  the  Cramer-Rao  lower  bound  and  the  correlator 
performance  estimate. 

T  (sec) _ S/N  (dB) 


2 

-1.70 

8 

-6.13 

32 

-9.68 

128 

-12.92 

512 

-16.05 

Table  1.  Required  Input  Signal-to-Noise  Ratio  for  P  *  10-6 


T  (sec) _ S/N  (dB) 


2 

-2.31 

8 

-6.57 

32 

-10.06 

128 

-13.29 

512 

-16.40 

Table  2.  Required  Input  Signal-to-Noise  Ratio  for  P  =  10~6 
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APPENDIX.  EVALUATION  OF  P  IN  (2) 


We  develop  (2)  as  follows: 

P  -  1  -  (dx  <|><x-a)  RUx)]^1  ( A— 1 ) 

6  *«• 

.  jdx  (fcx-ajfl-ffUx)]**1]  -  | 

o  +-^ 

.  f  dx  d(x-.){l-Ci(sx)]M-1j  ♦  J  dx  6(x-o)fl-[l-f(-Bx}3M-13  .  (A-2) 
-•0  0 

Form  ( A— 1 )  is  difficult  to  use,  because  It  requires  a  final  differencing  from 
1.  For  small  P  (the  case  of  Interest),  the  integral  in  (A-l)  requires  many 
digits  of  significance,  in  order  to  yield  accurate  P. 


In  (A-2),  observe  that  since  x  <  0  in  the  first  integral,  the  $  function 
values  there  will  be  <  1/2.  Also,  since  x  >  0  in  the  second  integral,  those  $ 
values  will  also  be  <  1/2.  Now  consider  the  two  functions 


M-l 


gfu}  i  1  -  u 


h  fu}  I  1  -  ( 1— u) 


for  0  <  u  <  1 


M-l 


(A-3) 


Then 

o 

P  »  j* dx  4(x-a)  gj$(Bx)}+  j  dx  b(x-a)  h[f(-ex)j  .  (A-4) 

— 0 

The  arguments  of  q{  }  and  hi  }  are  always  <  1/2.  For  evaluation  of  g,  we 
simply  use  the  definition  (A-3)  directly,  and  get  good  accuracy.  For 
evaluation  of  h,  we  must  use  a  power  series  expansion  about  u  =>  0  when  u  is 
small.  We  have  then 


(A-5) 


A-l 
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For  larger  u  |say  u  >  we  use  the  definition  (A— 3 )  directly.  So  h  can  be 

accurately  evaluated  very  easily.  Now  Pisa  sum  of  positive  quantities, 
where  all  the  quantities  can  be  accurately  evaluated. 

The  direct  definition  of  h  yields  the  alternative 


h{u]  *  l-(l-u)M“*  -  1-  expC (M— 1  )Jtn ( 1-u ) ] 


1-  exp 


|— ( M— 1 )  |>  un/n 

L  »=•  J 


(A-6) 


If  the  jfn  is  not  sufficiently  accurate  for  small  u  (like  1  / ( M— 1 ) ) ,  it  may  be 
necessary  to  resort  to  the  power  series  expansion  illustrated.  For  u  > 
accuracy  is  still  retained  in  the  difference  from  1. 

A  program  for  the  evaluation  of  P  and  then  a2  via  (I)  is  presented 
below. 
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1  ' 
ie 
2G 
30 
40 
50 
60 
70 
80 
90 
100 
110 
120 
130 
140 
150 
160 
170 
130 
190 
200 
210 
220 
230 
240 
250 
260 
270 
280 
290 
300 
310 
320 
330 
340 
350 
360 
370 
330 
3?r> 
40O 
4  1  0 
420 
430 
440 
450 
460 
470 
4  8  0 
490 
500 


TECH.  MEMO.  331065 
Db*- 1 6  ! 

Be  *  1 00  ' 

Tc-512  ! 

10*1/8  ! 
Mc*4*Bc*To  ! 

Sn*10"<.  l*Db>  ! 
Bt*Bc*Tc  ! 


APRIL  33 
10  LGTCS/N) 
BANDWIDTH  B 
OBSERVATION  TIME  T 
SEGMENT  LENGTH  To 
M 

S/N 

BT 


S2cr*. 375*< 1 +2*Sn)/<PI *PI *Sn*3n*Bt *Bc *Bc  > 


OUTPUT  0; "dB  ■  ";Db,"  B  ; Be , "  T  *"  ;Tc, 
OUTPUT  0;  "LGT<Si gm*CR)  =  " ;LGT<S8R<S2cr  ■ 
COM  A 1 , Bt , M 1 
Ra*Sn/< 1+Sn) 


B«*SQR<  1  +R**Ri) 

A1 *SQR<2*Bt >*R*/B* 


Ml-Mc-1 

L1*A1 ♦FNInwphi < IE-12) 
L1*MIN<-5| LI ) 

L2*FNPhi < -A  1 *B« ) 
L2*FNH<L2, Ml  > 

IF  L2>1E-12  THEN  230 
L2*5 

GOTO  240 


L2*A1 -FNInophi < 1E-12/L2> 
A*L  1 


To 


- " ;  7  o 
M  *";Mc 


3*<FNS1<A>+FNS1<B>>*.5 

N*2 

H*<  B-A ) * . 5 
F*<B-A>/3 
V1-9E99 
T  *0 

FOR  K* 1  TO  N-l  STEP  2 
T-T+FNS1  -:A  +  H*K) 

NEXT  K 
S*S  +  T 
Vo*V  1 

Vl*<SfT>*F 

PRINT  USING  "M. 12DE.7D ";V1,N 
IF  ABS<  V 1 -Vo  >  *  1 E - 3  THEN  440 
N  =  N«2 


H  =  H* .  5 
F  =  F  •*• .  5 
GOTO  310 
PRINT 
A  =  0 
B*L2 

S*v  FnS2  A  >  ♦  FNSi  (  B  >  >  * .  5 
N*2 

H= B-A  ■>  ■*  .  5 
F  = 1 E-A  •  3 


o^c 


9bV 7 


ov 


,„l'  •  ' 


.do«r<* 

-fVon 
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510  V2=8E8* 

526  T  *0 

530  FOR  K*  1  TO  1-1  STEP  2 
540  T  =  T+FNS2<'fli-H*K) 

550  NEXT  K 

560  S*S  +  T 

570  Vo=V2 

580  V2*(S+T>*F 

580  PRINT  USING  " M . 1 2DE , 7 D " ; V 2 , N 

600  IF  rtBS<V2-VoK*lE-8  THEN  650 

610  N-N*2 

620  H*H* . 5 

630  F=F*»5 

‘ 10  GOTO  520 

■£F>.  P»V  1+V2 

660  S2c  p«=P*T&*To' 3  +  C 1 -P>  *$2c  r 

670  OUTPUT  0;  "LGT <Si  gmiCPE)  =  "  ;  LGT SQR <  I- ;  pz 

680  END 

680  ! 

7O0  DEF  FNS 1 < X > 

710  COM  Ml ,B«,M1 

720  T 1  * . 38884223040 1 *EXP • - . 5*  < X-m 1 > -  2 1 
"SO  T2=FNPh i  Be  ♦X  > 

740  T2* 1 -T2  Ml 

750  RETURN  Tl*T2. 

760  FNEND 

770  i 

'  :0  DEF  FN3  ‘ 

y  COM  01,  .Ml 

O  *  1  ■ .  398*4228040 1  #EXP< - .  5* < X-fl -2? 

'*'10  f2*FNPhi 

820  T2*FNH'T2,M 

830  RETURN  T1*T 

3  -» ►?  FNEND 

350 

860  ».  FNM  •  ,  N  > 

370  IF  U'l  N  THEN  38; 

380  RETURN  1  - 1  - U  ■  N 
880  N 1  *  N  1 

8O0  S«T*H*U 

810  FOR  K«2  TO  N 

820  T  =  -T*<  N 1  -K  >  -»U  K 

830  3  =  3 +  T 

840  IF  flBS'  T  >  = 1 E- l 2  4hB8’ 3  ■  THEN  830 
850  HE  Y 

860  RETURN  3 

870  FNEND 

880  ' 


Cop? 

permit 
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990 

1000 

1010 

1020 

1030 

I  G40 
1050 
1060 
1070 
1030 
1090 
1100 
1  110 
1120 

I I  30 
1140 
1150 
1 160 
1170 
1130 
1190 
1200 
1210 
1220 
1230 
1240 
1250 
1260 
1270 
1280 
1290 
1300 
1310 
1  320 
1330 
1340 
1350 
1  360 
1  370 
1  330 
1390 
1400 
1410 


DEF  FNPhiOO 
INTEGER  J 

IF  ABSOO  >5.14  THEN  1270 
A*. 282342712475** 

C*C0S<A> 

S*SIN<A> 

B*C  +  0 
h=B*C- 1 

C-fl*a  .253675 IE- 18  +  B*7.  100O5E-20+A*?.  45 1 7E-2 1 
C*A*<1.533423425E-16+B*1 . 81 649277E- 1 7-0 > 
C*A*<1. 367684447 5 7E-14*E*1. 060 1364636E- 1 5+ 
C*fl*<8.89786526722E-13  +  B*S.  0606S838945E-1  -t.3  > 
C*A*<4. 2261614431  8E- 1 1  +  B *4. 46963229249E  - 1  2  -C.> 
C»A*<  1 . 466606 142 34E-9+B*l .  808485878 1 8E- 1  U-.-O  ■ 
C=A*<3. 72252349369E-8+B*5. 34275027608 E-9  +  0  ■ 
C*A*<6. 9 19275283 2 5E-7+B*l . 1 5330990944E- 7 *0 
C*A*<9. 43281  1 69338E-6+E* 1 . 320663 1 6364 E - 6  *  1  1 
C»ft* <9. 4490926881 0E-5+B*2. t 040458307 3E-5+C 
C*A*  <  6 . 97 1 83792408E-4+B*  1 . 7322  38162  5  5E - 4  *0  1 
C=A*<.  3. 80 1 58767985E-3  +  B*  1 .  18860645342E-3  *0 
C*A*< .8. '3935*26 157+B*. 0850790696 1228+0 
C  =  A*<- .  046‘'7 5 5234  •:25*B*.  01 72439625887 +C  • 

C«R*< . 1 03630245022* B*. 8439773381941 +C • 

0=A*(. 28 1339747265+B*. 03693 94549959+0 ■ 

C»A*< .33050152191 7*B*. 1 44227226362*0 1 
C*. 7032250O2744+B*.247255163140+C 
Phi *. 5*. O45015S158079*X+. 5*3*0 
RETURN  Phi 

IF  >0  7  THEN  RETURN  1 

N«MAX<6, 1  NT  <.  69  "ABS  <  X  >  ,  I  NTs  525-  •.  X*X  •  +1 

H=  1 
3*1 
B*  1  X 
0*B*  B 

FOR  J=1  TO  N 
A=<  l-.r-  J.'*fl*0 
3  *  3  *  A 
NEXT  J 

Ph  i  * .  39894228040 1  *E:IF  1  -  .  5*.  -HE  :•  E  ■  -  : 

IF  X/0  THEN  Ph  i  =  l-F'h i 

RETURN  Phi 

FNENB 


Gfopy  C.-r  ~:1 

penn1' 
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1420  DEF  FNInvphifZ)  1  invphi<z>  via  St  ■'e  col. 

1430  X=*2*Z-1 

1448  dim  H0:20>,fl<0:20^ 

1450  INTEGER  N 

1460  DATA  .  992885376619 ,  .  1204675 1 6 1  43,  .  3 1 6078 1 99'-'4*  i ,  .  00263670443716 
1470  DflTR  . 49963473024E-3, . 988982 186E -4 , . 2039 1 } i 276F-4 , . 4327271 6 2E-5 
1430  DATA  .  933031 4 IE-6, . 20673472E-6 ,  .  461597E--7  .  1 04 :6c 8E-- ,  .  237 1  5E- 3 
1490  DATA  . 54393E-9, . 12555E-9,  . 2914E-10, . 679E- 1 1 , . 159E-1 1, . 37E-12 
1500  DATA  .912158803413, 01 626628 1 8677 , .43353647* 95E-- j. . 2 1 443857O07E-3 
1510  DATA  . 262575108E-5, -. 302 1 09 1 05E-5 , -. 1  -40606E  ~ .. 624066 1 E-7 540 1 2E-9 
1520  DATA  -. 142321E-8, . 3438E-10, . 3338E-10, - . 1 46E- 1 l . - . 3 1 E- ! 2 , . 5E- 1 3 , . 2E-13 
1530  DATA  . 956679709020, -.0231070043091 ,-. 00437423609-51 , - . 57650342265E-3 
1540  DATA  -. 10961 0223 IE-4, . 25 1 08347O2E-4 , . 105623 3*0"c-4 , . 27S441233E-5 
1550  DATA  . 43243450E-6, -. 2053034E-7 , - . 4339 1 54E-7 . - . 1 76*40 1 E-7 . - . 395 1 29E-S 
1560  DATA  -.  13693E-9,  .27292E-9,  .  13232E-9,  .  3183E-10.  .  167E-1  1  ,  -.  2-34E-1 1 
1570  DATA  -.965E-12,-.22E-12 
1530  E  =  ABSOO 

1590  IF  ABSCX)  7*.  3  THEN  B»$QR<-L0G<4*2*<  1-Z  ■  '  ' 

1600  IF  AES C XX.  3  THEN  1700 
.610  IF  A  B  5  X  )  <  .  9  9  7  5  THEN  1660 
1620  Umax *20 
1630  RESTORE  1530 

1640  Y=- . 55945763 1 3304B+2. 2379 1 57 l 626 

1650  GOTO  1730 
1660  Nma>  *  1 5 
1670  RESTORE  1500 

1630  Y*-l .54331 3042374B+2. 565490 123 15 

1690  GOTO  1730 

1700  Nmax*13 

1710  RESTORE  1460 

1720  Y  *  X  *  X  *  3 . 125-1 

1730  PEDIM  A''0:  Nnt ax> 

1740  READ  A •'  *  .1 

1750  Y 2  =  Y  *  2 

1760  T 1 O > = 1 

1770  T'.  1  >  =  Y 

1730  FOR  N*2  TO  Nmax 

1790  T<N  * Y '7 •* T < N -  1  /-T<N-2> 

1800  NEXT  N 
1810  R * 0 

1820  FOR  N*Nf«  ax  TO  0  STEP  -1 
1830  R*R4fl(N'*TOO 
1840  NEXT  N 

1 850  I nvph i =SGN > X  •  *B*R* 1 . 4 1 42 1  356237 
I860  RETURN  Irvphi 
1870  FHEND 


A-6 
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